Jul 28 19:57:16.458451 ip-10-0-139-54 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 28 19:57:16.458465 ip-10-0-139-54 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 28 19:57:16.458475 ip-10-0-139-54 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 28 19:57:16.458834 ip-10-0-139-54 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 28 19:57:25.791111 ip-10-0-139-54 systemd[1]: kubelet.service: Failed to schedule restart job: Unit openvswitch.service not found.
Jul 28 19:57:25.791131 ip-10-0-139-54 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 33e978771b4444e8b424c12d2cf7f47b --
Jul 28 19:58:55.985845 ip-10-0-139-54 systemd[1]: Starting Kubernetes Kubelet...
Jul 28 19:58:56.492920 ip-10-0-139-54 kubenswrapper[2554]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 19:58:56.492920 ip-10-0-139-54 kubenswrapper[2554]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 28 19:58:56.492920 ip-10-0-139-54 kubenswrapper[2554]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 19:58:56.492920 ip-10-0-139-54 kubenswrapper[2554]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 28 19:58:56.492920 ip-10-0-139-54 kubenswrapper[2554]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 19:58:56.493839 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.493751    2554 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 28 19:58:56.498357 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498340    2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 19:58:56.498357 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498358    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 19:58:56.498427 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498362    2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 19:58:56.498427 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498365    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 19:58:56.498427 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498368    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 19:58:56.498427 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498371    2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 19:58:56.498427 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498374    2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 19:58:56.498427 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498377    2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 19:58:56.498427 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498379    2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 19:58:56.498427 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498382    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 19:58:56.498427 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498385    2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 19:58:56.498427 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498387    2554 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 19:58:56.498427 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498390    2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 19:58:56.498427 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498393    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 19:58:56.498427 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498395    2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 19:58:56.498427 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498398    2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 19:58:56.498427 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498401    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 19:58:56.498427 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498406    2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 19:58:56.498427 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498409    2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 19:58:56.498427 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498411    2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 19:58:56.498427 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498413    2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 19:58:56.498931 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498418    2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 19:58:56.498931 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498422    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 19:58:56.498931 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498425    2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 19:58:56.498931 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498428    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 19:58:56.498931 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498431    2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 19:58:56.498931 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498434    2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 19:58:56.498931 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498436    2554 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 19:58:56.498931 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498439    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 19:58:56.498931 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498443    2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 19:58:56.498931 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498445    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 19:58:56.498931 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498448    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 19:58:56.498931 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498451    2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 19:58:56.498931 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498453    2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 19:58:56.498931 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498456    2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 19:58:56.498931 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498458    2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 19:58:56.498931 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498461    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 19:58:56.498931 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498463    2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 19:58:56.498931 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498466    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 19:58:56.498931 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498468    2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 19:58:56.499383 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498470    2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 19:58:56.499383 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498473    2554 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 19:58:56.499383 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498476    2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 19:58:56.499383 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498478    2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 19:58:56.499383 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498481    2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 19:58:56.499383 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498483    2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 19:58:56.499383 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498486    2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 19:58:56.499383 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498488    2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 19:58:56.499383 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498491    2554 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 19:58:56.499383 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498494    2554 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 19:58:56.499383 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498497    2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 19:58:56.499383 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498499    2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 19:58:56.499383 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498502    2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 19:58:56.499383 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498505    2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 19:58:56.499383 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498508    2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 19:58:56.499383 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498511    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 19:58:56.499383 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498513    2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 19:58:56.499383 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498516    2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 19:58:56.499383 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498518    2554 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 19:58:56.499383 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498522    2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 19:58:56.499884 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498525    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 19:58:56.499884 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498528    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 19:58:56.499884 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498531    2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 19:58:56.499884 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498533    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 19:58:56.499884 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498536    2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 19:58:56.499884 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498538    2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 19:58:56.499884 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498540    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 19:58:56.499884 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498556    2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 19:58:56.499884 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498559    2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 19:58:56.499884 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498562    2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 19:58:56.499884 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498565    2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 19:58:56.499884 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498573    2554 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 19:58:56.499884 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498576    2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 19:58:56.499884 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498578    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 19:58:56.499884 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498581    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 19:58:56.499884 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498583    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 19:58:56.499884 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498588    2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 19:58:56.499884 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498591    2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 19:58:56.499884 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498594    2554 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 19:58:56.500335 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498597    2554 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 19:58:56.500335 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498600    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 19:58:56.500335 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498603    2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 19:58:56.500335 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498607    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 19:58:56.500335 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498610    2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 19:58:56.500335 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498613    2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 19:58:56.500335 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498616    2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 19:58:56.500335 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498620    2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 19:58:56.500335 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498623    2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 19:58:56.500335 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498626    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 19:58:56.500335 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498629    2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 19:58:56.500335 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498631    2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 19:58:56.500335 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498634    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 19:58:56.500335 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498636    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 19:58:56.500335 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498639    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 19:58:56.500335 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498641    2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 19:58:56.500335 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498644    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 19:58:56.500335 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498646    2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 19:58:56.500335 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498648    2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 19:58:56.500335 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.498651    2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 19:58:56.500845 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499368    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 19:58:56.500845 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499374    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 19:58:56.500845 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499377    2554 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 19:58:56.500845 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499379    2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 19:58:56.500845 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499383    2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 19:58:56.500845 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499386    2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 19:58:56.500845 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499389    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 19:58:56.500845 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499392    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 19:58:56.500845 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499395    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 19:58:56.500845 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499397    2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 19:58:56.500845 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499400    2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 19:58:56.500845 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499403    2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 19:58:56.500845 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499405    2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 19:58:56.500845 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499408    2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 19:58:56.500845 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499411    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 19:58:56.500845 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499413    2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 19:58:56.500845 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499416    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 19:58:56.500845 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499418    2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 19:58:56.500845 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499421    2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 19:58:56.500845 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499424    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 19:58:56.501339 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499426    2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 19:58:56.501339 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499428    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 19:58:56.501339 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499431    2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 19:58:56.501339 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499433    2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 19:58:56.501339 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499436    2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 19:58:56.501339 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499438    2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 19:58:56.501339 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499441    2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 19:58:56.501339 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499443    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 19:58:56.501339 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499446    2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 19:58:56.501339 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499448    2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 19:58:56.501339 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499450    2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 19:58:56.501339 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499453    2554 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 19:58:56.501339 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499455    2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 19:58:56.501339 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499458    2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 19:58:56.501339 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499462    2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 19:58:56.501339 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499464    2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 19:58:56.501339 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499468    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 19:58:56.501339 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499470    2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 19:58:56.501339 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499473    2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 19:58:56.501822 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499476    2554 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 19:58:56.501822 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499479    2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 19:58:56.501822 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499481    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 19:58:56.501822 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499484    2554 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 19:58:56.501822 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499486    2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 19:58:56.501822 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499489    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 19:58:56.501822 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499491    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 19:58:56.501822 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499493    2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 19:58:56.501822 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499496    2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 19:58:56.501822 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499498    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 19:58:56.501822 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499501    2554 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 19:58:56.501822 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499503    2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 19:58:56.501822 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499506    2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 19:58:56.501822 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499508    2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 19:58:56.501822 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499510    2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 19:58:56.501822 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499513    2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 19:58:56.501822 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499516    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 19:58:56.501822 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499519    2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 19:58:56.501822 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499521    2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 19:58:56.501822 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499523    2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 19:58:56.502294 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499526    2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 19:58:56.502294 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499528    2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 19:58:56.502294 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499530    2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 19:58:56.502294 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499533    2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 19:58:56.502294 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499537    2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 19:58:56.502294 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499540    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 19:58:56.502294 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499559    2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 19:58:56.502294 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499564    2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 19:58:56.502294 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499567    2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 19:58:56.502294 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499570    2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 19:58:56.502294 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499572    2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 19:58:56.502294 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499576    2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 19:58:56.502294 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499579    2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 19:58:56.502294 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499582    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 19:58:56.502294 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499584    2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 19:58:56.502294 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499587    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 19:58:56.502294 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499589    2554 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 19:58:56.502294 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499592    2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 19:58:56.502294 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499594    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 19:58:56.502782 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499597    2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 19:58:56.502782 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499599    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 19:58:56.502782 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499602    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 19:58:56.502782 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499604    2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 19:58:56.502782 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499608    2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 19:58:56.502782 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499612    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 19:58:56.502782 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499615    2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 19:58:56.502782 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499617    2554 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 19:58:56.502782 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499620    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 19:58:56.502782 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499623    2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 19:58:56.502782 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499626    2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 19:58:56.502782 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499628    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 19:58:56.502782 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499631    2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 19:58:56.502782 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499633    2554 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 19:58:56.502782 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499637    2554 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 19:58:56.502782 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499639    2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 19:58:56.502782 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499642    2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 19:58:56.502782 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499645    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 19:58:56.502782 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499647    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 19:58:56.503245 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499650    2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 19:58:56.503245 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.499652    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 19:58:56.503245 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500011    2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 19:58:56.503245 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500016    2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 19:58:56.503245 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500019    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 19:58:56.503245 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500021    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 19:58:56.503245 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500025    2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 19:58:56.503245 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500028    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 19:58:56.503245 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500030    2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 19:58:56.503245 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500033    2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 19:58:56.503245 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500036    2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 19:58:56.503245 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500038    2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 19:58:56.503245 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500041    2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 19:58:56.503245 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500046    2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 19:58:56.503245 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500049    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 19:58:56.503245 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500052    2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 19:58:56.503245 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500055    2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 19:58:56.503245 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500058    2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 19:58:56.503245 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500062    2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 19:58:56.503715 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500065    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 19:58:56.503715 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500068    2554 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 19:58:56.503715 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500070    2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 19:58:56.503715 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500073    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 19:58:56.503715 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500075    2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 19:58:56.503715 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500078    2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 19:58:56.503715 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500080    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 19:58:56.503715 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500083    2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 19:58:56.503715 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500085    2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 19:58:56.503715 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500088    2554 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 19:58:56.503715 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500091    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 19:58:56.503715 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500094    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 19:58:56.503715 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500096    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 19:58:56.503715 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500099    2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 19:58:56.503715 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500101    2554 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 19:58:56.503715 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500103    2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 19:58:56.503715 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500106    2554 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 19:58:56.503715 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500108    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 19:58:56.503715 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500111    2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 19:58:56.503715 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500114    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 19:58:56.504226 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500118    2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 19:58:56.504226 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500121    2554 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 19:58:56.504226 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500123    2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 19:58:56.504226 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500125    2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 19:58:56.504226 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500128    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 19:58:56.504226 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500130    2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 19:58:56.504226 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500133    2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 19:58:56.504226 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500135    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 19:58:56.504226 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500138    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 19:58:56.504226 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500140    2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 19:58:56.504226 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500143    2554 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 19:58:56.504226 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500146    2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 19:58:56.504226 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500149    2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 19:58:56.504226 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500152    2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 19:58:56.504226 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500154    2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 19:58:56.504226 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500156    2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 19:58:56.504226 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500159    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 19:58:56.504226 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500161    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 19:58:56.504226 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500164    2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 19:58:56.504696 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500166    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 19:58:56.504696 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500169    2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 19:58:56.504696 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500172    2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 19:58:56.504696 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500174    2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 19:58:56.504696 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500177    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 19:58:56.504696 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500179    2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 19:58:56.504696 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500182    2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 19:58:56.504696 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500185    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 19:58:56.504696 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500187    2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 19:58:56.504696 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500189    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 19:58:56.504696 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500192    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 19:58:56.504696 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500194    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 19:58:56.504696 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500197    2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 19:58:56.504696 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500200    2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 19:58:56.504696 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500203    2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 19:58:56.504696 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500205    2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 19:58:56.504696 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500207    2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 19:58:56.504696 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500210    2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 19:58:56.504696 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500212    2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 19:58:56.504696 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500215    2554 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 19:58:56.505181 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500217    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 19:58:56.505181 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500219    2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 19:58:56.505181 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500222    2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 19:58:56.505181 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500227    2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 19:58:56.505181 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500229    2554 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 19:58:56.505181 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500231    2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 19:58:56.505181 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500234    2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 19:58:56.505181 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500236    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 19:58:56.505181 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500239    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 19:58:56.505181 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500241    2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 19:58:56.505181 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500243    2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 19:58:56.505181 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500246    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 19:58:56.505181 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500248    2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 19:58:56.505181 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500251    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 19:58:56.505181 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500253    2554 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 19:58:56.505181 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500256    2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 19:58:56.505181 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500258    2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 19:58:56.505181 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500261    2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 19:58:56.505181 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500264    2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 19:58:56.505181 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500266    2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 19:58:56.505728 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500268    2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 19:58:56.505728 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500271    2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 19:58:56.505728 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500273    2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 19:58:56.505728 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500348    2554 flags.go:64] FLAG: --address="0.0.0.0"
Jul 28 19:58:56.505728 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500354    2554 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 28 19:58:56.505728 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500360    2554 flags.go:64] FLAG: --anonymous-auth="true"
Jul 28 19:58:56.505728 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500364    2554 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 28 19:58:56.505728 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500370    2554 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 28 19:58:56.505728 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500373    2554 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 28 19:58:56.505728 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500378    2554 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 28 19:58:56.505728 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500382    2554 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 28 19:58:56.505728 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500385    2554 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 28 19:58:56.505728 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500388    2554 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 28 19:58:56.505728 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500391    2554 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 28 19:58:56.505728 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500395    2554 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 28 19:58:56.505728 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500399    2554 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 28 19:58:56.505728 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500402    2554 flags.go:64] FLAG: --cgroup-root=""
Jul 28 19:58:56.505728 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500405    2554 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 28 19:58:56.505728 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500409    2554 flags.go:64] FLAG: --client-ca-file=""
Jul 28 19:58:56.505728 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500412    2554 flags.go:64] FLAG: --cloud-provider="external"
Jul 28 19:58:56.505728 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500414    2554 flags.go:64] FLAG: --cluster-dns="[]"
Jul 28 19:58:56.505728 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500419    2554 flags.go:64] FLAG: --cluster-domain=""
Jul 28 19:58:56.505728 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500422    2554 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 28 19:58:56.506276 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500425    2554 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 28 19:58:56.506276 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500428    2554 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 28 19:58:56.506276 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500432    2554 flags.go:64] FLAG: --container-log-max-files="5"
Jul 28 19:58:56.506276 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500436    2554 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 28 19:58:56.506276 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500439    2554 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 28 19:58:56.506276 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500442    2554 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 28 19:58:56.506276 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500445    2554 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 28 19:58:56.506276 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500449    2554 flags.go:64] FLAG: --contention-profiling="false"
Jul 28 19:58:56.506276 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500452    2554 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 28 19:58:56.506276 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500455    2554 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 28 19:58:56.506276 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500457    2554 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 28 19:58:56.506276 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500460    2554 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 28 19:58:56.506276 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500465    2554 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 28 19:58:56.506276 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500467    2554 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 28 19:58:56.506276 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500470    2554 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 28 19:58:56.506276 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500473    2554 flags.go:64] FLAG: --enable-load-reader="false"
Jul 28 19:58:56.506276 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500476    2554 flags.go:64] FLAG: --enable-server="true"
Jul 28 19:58:56.506276 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500479    2554 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 28 19:58:56.506276 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500483    2554 flags.go:64] FLAG: --event-burst="100"
Jul 28 19:58:56.506276 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500486    2554 flags.go:64] FLAG: --event-qps="50"
Jul 28 19:58:56.506276 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500489    2554 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 28 19:58:56.506276 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500492    2554 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 28 19:58:56.506276 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500495    2554 flags.go:64] FLAG: --eviction-hard=""
Jul 28 19:58:56.506276 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500498    2554 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 28 19:58:56.506880 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500502    2554 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 28 19:58:56.506880 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500505    2554 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 28 19:58:56.506880 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500508    2554 flags.go:64] FLAG: --eviction-soft=""
Jul 28 19:58:56.506880 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500511    2554 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 28 19:58:56.506880 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500514    2554 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 28 19:58:56.506880 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500517    2554 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 28 19:58:56.506880 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500519    2554 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 28 19:58:56.506880 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500522    2554 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 28 19:58:56.506880 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500526    2554 flags.go:64] FLAG: --fail-swap-on="true"
Jul 28 19:58:56.506880 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500528    2554 flags.go:64] FLAG: --feature-gates=""
Jul 28 19:58:56.506880 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500532    2554 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 28 19:58:56.506880 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500535    2554 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 28 19:58:56.506880 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500538    2554 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 28 19:58:56.506880 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500541    2554 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 28 19:58:56.506880 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500556    2554 flags.go:64] FLAG: --healthz-port="10248"
Jul 28 19:58:56.506880 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500559    2554 flags.go:64] FLAG: --help="false"
Jul 28 19:58:56.506880 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500562    2554 flags.go:64] FLAG: --hostname-override="ip-10-0-139-54.ec2.internal"
Jul 28 19:58:56.506880 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500565    2554 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 28 19:58:56.506880 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500568    2554 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 28 19:58:56.506880 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500571    2554 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 28 19:58:56.506880 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500574    2554 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 28 19:58:56.506880 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500578    2554 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 28 19:58:56.506880 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500581    2554 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 28 19:58:56.506880 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500583    2554 flags.go:64] FLAG: --image-service-endpoint=""
Jul 28 19:58:56.507480 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500586    2554 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 28 19:58:56.507480 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500589    2554 flags.go:64] FLAG: --kube-api-burst="100"
Jul 28 19:58:56.507480 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500592    2554 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 28 19:58:56.507480 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500596    2554 flags.go:64] FLAG: --kube-api-qps="50"
Jul 28 19:58:56.507480 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500598    2554 flags.go:64] FLAG: --kube-reserved=""
Jul 28 19:58:56.507480 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500601    2554 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 28 19:58:56.507480 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500604    2554 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 28 19:58:56.507480 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500607    2554 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 28 19:58:56.507480 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500611    2554 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 28 19:58:56.507480 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500614    2554 flags.go:64] FLAG: --lock-file=""
Jul 28 19:58:56.507480 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500617    2554 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 28 19:58:56.507480 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500620    2554 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 28 19:58:56.507480 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500622    2554 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 28 19:58:56.507480 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500628    2554 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 28 19:58:56.507480 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500631    2554 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 28 19:58:56.507480 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500634    2554 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 28 19:58:56.507480 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500637    2554 flags.go:64] FLAG: --logging-format="text"
Jul 28 19:58:56.507480 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500640    2554 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 28 19:58:56.507480 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500643    2554 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 28 19:58:56.507480 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500646    2554 flags.go:64] FLAG: --manifest-url=""
Jul 28 19:58:56.507480 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500649    2554 flags.go:64] FLAG: --manifest-url-header=""
Jul 28 19:58:56.507480 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500653    2554 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 28 19:58:56.507480 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500656    2554 flags.go:64] FLAG: --max-open-files="1000000"
Jul 28 19:58:56.507480 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500660    2554 flags.go:64] FLAG: --max-pods="110"
Jul 28 19:58:56.507480 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500663    2554 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 28 19:58:56.508110 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500666    2554 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 28 19:58:56.508110 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500669    2554 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 28 19:58:56.508110 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500673    2554 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 28 19:58:56.508110 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500676    2554 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 28 19:58:56.508110 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500679    2554 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 28 19:58:56.508110 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500682    2554 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 28 19:58:56.508110 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500689    2554 flags.go:64] FLAG: --node-status-max-images="50"
Jul 28 19:58:56.508110 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500692    2554 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 28 19:58:56.508110 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500695    2554 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 28 19:58:56.508110 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500698    2554 flags.go:64] FLAG: --pod-cidr=""
Jul 28 19:58:56.508110 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500702    2554 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 28 19:58:56.508110 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500707    2554 flags.go:64] FLAG: --pod-manifest-path=""
Jul 28 19:58:56.508110 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500710    2554 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 28 19:58:56.508110 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500713    2554 flags.go:64] FLAG: --pods-per-core="0"
Jul 28 19:58:56.508110 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500716    2554 flags.go:64] FLAG: --port="10250"
Jul 28 19:58:56.508110 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500720    2554 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 28 19:58:56.508110 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500723    2554 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0b53b7c6502b1b024"
Jul 28 19:58:56.508110 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500727    2554 flags.go:64] FLAG: --qos-reserved=""
Jul 28 19:58:56.508110 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500730    2554 flags.go:64] FLAG: --read-only-port="10255"
Jul 28 19:58:56.508110 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500733    2554 flags.go:64] FLAG: --register-node="true"
Jul 28 19:58:56.508110 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500736    2554 flags.go:64] FLAG: --register-with-taints=""
Jul 28 19:58:56.508110 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500740    2554 flags.go:64] FLAG: --registry-burst="10"
Jul 28 19:58:56.508110 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500743    2554 flags.go:64] FLAG: --registry-qps="5"
Jul 28 19:58:56.508110 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500746    2554 flags.go:64] FLAG: --reserved-cpus=""
Jul 28 19:58:56.508691 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500748    2554 flags.go:64] FLAG: --reserved-memory=""
Jul 28 19:58:56.508691 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500752    2554 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 28 19:58:56.508691 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500755    2554 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 28 19:58:56.508691 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500758    2554 flags.go:64] FLAG: --rotate-certificates="false"
Jul 28 19:58:56.508691 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500761    2554 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 28 19:58:56.508691 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500763    2554 flags.go:64] FLAG: --runonce="false"
Jul 28 19:58:56.508691 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500766    2554 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 28 19:58:56.508691 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500769    2554 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 28 19:58:56.508691 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500772    2554 flags.go:64] FLAG: --seccomp-default="false"
Jul 28 19:58:56.508691 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500776    2554 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 28 19:58:56.508691 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500778    2554 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 28 19:58:56.508691 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500782    2554 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 28 19:58:56.508691 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500785    2554 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 28 19:58:56.508691 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500788    2554 flags.go:64] FLAG: --storage-driver-password="root"
Jul 28 19:58:56.508691 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500791    2554 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 28 19:58:56.508691 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500793    2554 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 28 19:58:56.508691 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500796    2554 flags.go:64] FLAG: --storage-driver-user="root"
Jul 28 19:58:56.508691 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500799    2554 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 28 19:58:56.508691 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500802    2554 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 28 19:58:56.508691 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500805    2554 flags.go:64] FLAG: --system-cgroups=""
Jul 28 19:58:56.508691 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500808    2554 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 28 19:58:56.508691 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500813    2554 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 28 19:58:56.508691 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500816    2554 flags.go:64] FLAG: --tls-cert-file=""
Jul 28 19:58:56.508691 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500820    2554 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 28 19:58:56.508691 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500824    2554 flags.go:64] FLAG: --tls-min-version=""
Jul 28 19:58:56.509335 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500828    2554 flags.go:64] FLAG: --tls-private-key-file=""
Jul 28 19:58:56.509335 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500831    2554 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 28 19:58:56.509335 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500834    2554 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 28 19:58:56.509335 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500837    2554 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 28 19:58:56.509335 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500840    2554 flags.go:64] FLAG: --v="2"
Jul 28 19:58:56.509335 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500843    2554 flags.go:64] FLAG: --version="false"
Jul 28 19:58:56.509335 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500847    2554 flags.go:64] FLAG: --vmodule=""
Jul 28 19:58:56.509335 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500851    2554 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 28 19:58:56.509335 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.500855    2554 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 28 19:58:56.509335 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500965    2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 19:58:56.509335 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500968    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 19:58:56.509335 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500971    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 19:58:56.509335 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500974    2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 19:58:56.509335 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500978    2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 19:58:56.509335 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500982    2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 19:58:56.509335 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500985    2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 19:58:56.509335 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500988    2554 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 19:58:56.509335 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500991    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 19:58:56.509335 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500994    2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 19:58:56.509335 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500996    2554 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 19:58:56.509335 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.500999    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 19:58:56.509893 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501002    2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 19:58:56.509893 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501005    2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 19:58:56.509893 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501008    2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 19:58:56.509893 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501012    2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 19:58:56.509893 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501015    2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 19:58:56.509893 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501018    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 19:58:56.509893 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501021    2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 19:58:56.509893 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501023    2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 19:58:56.509893 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501026    2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 19:58:56.509893 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501029    2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 19:58:56.509893 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501032    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 19:58:56.509893 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501036    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 19:58:56.509893 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501039    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 19:58:56.509893 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501041    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 19:58:56.509893 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501044    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 19:58:56.509893 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501046    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 19:58:56.509893 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501049    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 19:58:56.509893 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501053    2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 19:58:56.509893 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501056    2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 19:58:56.510418 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501059    2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 19:58:56.510418 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501062    2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 19:58:56.510418 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501065    2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 19:58:56.510418 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501067    2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 19:58:56.510418 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501070    2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 19:58:56.510418 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501072    2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 19:58:56.510418 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501075    2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 19:58:56.510418 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501078    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 19:58:56.510418 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501080    2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 19:58:56.510418 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501083    2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 19:58:56.510418 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501085    2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 19:58:56.510418 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501088    2554 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 19:58:56.510418 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501091    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 19:58:56.510418 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501094    2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 19:58:56.510418 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501096    2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 19:58:56.510418 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501099    2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 19:58:56.510418 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501101    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 19:58:56.510418 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501104    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 19:58:56.510418 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501107    2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 19:58:56.510418 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501109    2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 19:58:56.510940 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501112    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 19:58:56.510940 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501115    2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 19:58:56.510940 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501118    2554 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 19:58:56.510940 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501120    2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 19:58:56.510940 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501124    2554 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 19:58:56.510940 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501127    2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 19:58:56.510940 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501130    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 19:58:56.510940 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501133    2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 19:58:56.510940 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501135    2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 19:58:56.510940 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501138    2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 19:58:56.510940 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501140    2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 19:58:56.510940 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501143    2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 19:58:56.510940 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501146    2554 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 19:58:56.510940 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501148    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 19:58:56.510940 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501151    2554 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 19:58:56.510940 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501153    2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 19:58:56.510940 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501156    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 19:58:56.510940 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501158    2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 19:58:56.510940 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501161    2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 19:58:56.510940 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501164    2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 19:58:56.511424 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501166    2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 19:58:56.511424 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501169    2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 19:58:56.511424 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501172    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 19:58:56.511424 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501174    2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 19:58:56.511424 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501177    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 19:58:56.511424 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501179    2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 19:58:56.511424 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501182    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 19:58:56.511424 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501185    2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 19:58:56.511424 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501187    2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 19:58:56.511424 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501190    2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 19:58:56.511424 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501192    2554 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 19:58:56.511424 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501195    2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 19:58:56.511424 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501198    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 19:58:56.511424 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501201    2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 19:58:56.511424 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501203    2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 19:58:56.511424 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501206    2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 19:58:56.511424 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501210    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 19:58:56.511424 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501213    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 19:58:56.511424 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501216    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 19:58:56.511424 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501218    2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 19:58:56.511991 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501221    2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 19:58:56.511991 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501224    2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 19:58:56.511991 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501233    2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 19:58:56.511991 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501237    2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 19:58:56.511991 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501239    2554 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 19:58:56.511991 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501242    2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 19:58:56.511991 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501244    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 19:58:56.511991 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.501247    2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 19:58:56.511991 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.504444    2554 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 19:58:56.511991 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.511153    2554 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 28 19:58:56.511991 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.511168    2554 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 28 19:58:56.511991 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.511181    2554 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 28 19:58:56.511991 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.511187    2554 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 28 19:58:56.511991 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511243    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 19:58:56.512489 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511247    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 19:58:56.512489 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511251    2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 19:58:56.512489 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511254    2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 19:58:56.512489 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511256    2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 19:58:56.512489 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511261    2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 19:58:56.512489 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511265    2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 19:58:56.512489 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511268    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 19:58:56.512489 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511272    2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 19:58:56.512489 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511275    2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 19:58:56.512489 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511277    2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 19:58:56.512489 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511280    2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 19:58:56.512489 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511283    2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 19:58:56.512489 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511285    2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 19:58:56.512489 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511288    2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 19:58:56.512489 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511291    2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 19:58:56.512489 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511293    2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 19:58:56.512489 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511296    2554 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 19:58:56.512489 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511298    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 19:58:56.512489 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511301    2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 19:58:56.512489 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511304    2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 19:58:56.513277 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511306    2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 19:58:56.513277 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511309    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 19:58:56.513277 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511312    2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 19:58:56.513277 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511314    2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 19:58:56.513277 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511317    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 19:58:56.513277 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511319    2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 19:58:56.513277 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511322    2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 19:58:56.513277 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511325    2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 19:58:56.513277 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511329    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 19:58:56.513277 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511332    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 19:58:56.513277 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511335    2554 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 19:58:56.513277 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511337    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 19:58:56.513277 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511340    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 19:58:56.513277 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511343    2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 19:58:56.513277 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511345    2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 19:58:56.513277 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511348    2554 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 19:58:56.513277 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511350    2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 19:58:56.513277 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511353    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 19:58:56.513277 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511355    2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 19:58:56.514061 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511357    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 19:58:56.514061 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511360    2554 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 19:58:56.514061 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511362    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 19:58:56.514061 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511365    2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 19:58:56.514061 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511367    2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 19:58:56.514061 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511370    2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 19:58:56.514061 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511372    2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 19:58:56.514061 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511375    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 19:58:56.514061 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511377    2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 19:58:56.514061 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511380    2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 19:58:56.514061 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511382    2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 19:58:56.514061 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511385    2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 19:58:56.514061 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511388    2554 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 19:58:56.514061 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511390    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 19:58:56.514061 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511393    2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 19:58:56.514061 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511395    2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 19:58:56.514061 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511397    2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 19:58:56.514061 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511400    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 19:58:56.514061 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511403    2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 19:58:56.514061 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511405    2554 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 19:58:56.514678 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511408    2554 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 19:58:56.514678 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511411    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 19:58:56.514678 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511415    2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 19:58:56.514678 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511417    2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 19:58:56.514678 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511420    2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 19:58:56.514678 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511423    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 19:58:56.514678 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511425    2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 19:58:56.514678 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511428    2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 19:58:56.514678 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511431    2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 19:58:56.514678 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511434    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 19:58:56.514678 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511436    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 19:58:56.514678 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511439    2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 19:58:56.514678 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511441    2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 19:58:56.514678 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511444    2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 19:58:56.514678 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511446    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 19:58:56.514678 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511449    2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 19:58:56.514678 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511451    2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 19:58:56.514678 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511455    2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 19:58:56.514678 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511458    2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 19:58:56.515133 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511461    2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 19:58:56.515133 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511464    2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 19:58:56.515133 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511466    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 19:58:56.515133 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511469    2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 19:58:56.515133 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511471    2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 19:58:56.515133 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511474    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 19:58:56.515133 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511476    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 19:58:56.515133 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511480    2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 19:58:56.515133 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511483    2554 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 19:58:56.515133 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511487    2554 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 19:58:56.515133 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511490    2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 19:58:56.515133 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511492    2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 19:58:56.515133 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511495    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 19:58:56.515133 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511498    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 19:58:56.515133 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511501    2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 19:58:56.515133 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511504    2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 19:58:56.515133 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511507    2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 19:58:56.515133 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511509    2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 19:58:56.515133 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511512    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 19:58:56.515133 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511514    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 19:58:56.515764 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.511519    2554 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 19:58:56.515764 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511648    2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 19:58:56.515764 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511654    2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 19:58:56.515764 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511657    2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 19:58:56.515764 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511659    2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 19:58:56.515764 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511662    2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 19:58:56.515764 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511665    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 19:58:56.515764 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511667    2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 19:58:56.515764 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511669    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 19:58:56.515764 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511672    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 19:58:56.515764 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511675    2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 19:58:56.515764 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511677    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 19:58:56.515764 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511680    2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 19:58:56.515764 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511683    2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 19:58:56.516114 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511686    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 19:58:56.516114 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511688    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 19:58:56.516114 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511690    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 19:58:56.516114 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511693    2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 19:58:56.516114 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511695    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 19:58:56.516114 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511698    2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 19:58:56.516114 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511700    2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 19:58:56.516114 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511703    2554 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 19:58:56.516114 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511705    2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 19:58:56.516114 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511707    2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 19:58:56.516114 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511710    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 19:58:56.516114 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511712    2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 19:58:56.516114 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511715    2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 19:58:56.516114 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511718    2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 19:58:56.516114 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511720    2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 19:58:56.516114 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511723    2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 19:58:56.516114 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511726    2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 19:58:56.516114 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511729    2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 19:58:56.516114 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511734    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 19:58:56.516114 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511736    2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 19:58:56.516600 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511738    2554 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 19:58:56.516600 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511741    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 19:58:56.516600 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511744    2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 19:58:56.516600 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511746    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 19:58:56.516600 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511749    2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 19:58:56.516600 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511751    2554 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 19:58:56.516600 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511753    2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 19:58:56.516600 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511756    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 19:58:56.516600 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511758    2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 19:58:56.516600 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511761    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 19:58:56.516600 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511763    2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 19:58:56.516600 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511765    2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 19:58:56.516600 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511768    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 19:58:56.516600 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511770    2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 19:58:56.516600 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511773    2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 19:58:56.516600 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511775    2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 19:58:56.516600 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511778    2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 19:58:56.516600 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511780    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 19:58:56.516600 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511783    2554 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 19:58:56.516600 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511785    2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 19:58:56.517109 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511787    2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 19:58:56.517109 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511790    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 19:58:56.517109 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511792    2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 19:58:56.517109 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511795    2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 19:58:56.517109 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511798    2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 19:58:56.517109 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511801    2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 19:58:56.517109 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511804    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 19:58:56.517109 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511806    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 19:58:56.517109 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511809    2554 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 19:58:56.517109 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511811    2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 19:58:56.517109 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511814    2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 19:58:56.517109 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511818    2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 19:58:56.517109 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511821    2554 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 19:58:56.517109 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511824    2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 19:58:56.517109 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511826    2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 19:58:56.517109 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511829    2554 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 19:58:56.517109 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511832    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 19:58:56.517109 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511835    2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 19:58:56.517109 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511837    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 19:58:56.517604 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511840    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 19:58:56.517604 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511842    2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 19:58:56.517604 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511845    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 19:58:56.517604 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511847    2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 19:58:56.517604 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511850    2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 19:58:56.517604 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511852    2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 19:58:56.517604 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511854    2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 19:58:56.517604 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511857    2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 19:58:56.517604 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511859    2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 19:58:56.517604 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511862    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 19:58:56.517604 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511865    2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 19:58:56.517604 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511868    2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 19:58:56.517604 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511871    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 19:58:56.517604 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511874    2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 19:58:56.517604 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511877    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 19:58:56.517604 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511880    2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 19:58:56.517604 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511883    2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 19:58:56.517604 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511885    2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 19:58:56.517604 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511888    2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 19:58:56.518053 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511891    2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 19:58:56.518053 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511893    2554 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 19:58:56.518053 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511896    2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 19:58:56.518053 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511899    2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 19:58:56.518053 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511902    2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 19:58:56.518053 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511904    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 19:58:56.518053 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511907    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 19:58:56.518053 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:56.511909    2554 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 19:58:56.518053 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.511914    2554 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 19:58:56.518053 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.512037    2554 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 28 19:58:56.518053 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.515278    2554 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 28 19:58:56.518053 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.516847    2554 server.go:1019] "Starting client certificate rotation"
Jul 28 19:58:56.518053 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.516946    2554 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 19:58:56.518053 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.516985    2554 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 19:58:56.522589 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.522574    2554 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 19:58:56.523937 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.523919    2554 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 19:58:56.532445 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.532430    2554 log.go:25] "Validated CRI v1 runtime API"
Jul 28 19:58:56.534557 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.534537    2554 log.go:25] "Validated CRI v1 image API"
Jul 28 19:58:56.535316 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.535290    2554 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 28 19:58:56.538149 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.538133    2554 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 fbfc65e2-d37f-4d55-b629-67c5d88af83a:/dev/nvme0n1p3 fe98826c-ec00-424a-9380-b9a49181dc9d:/dev/nvme0n1p4]
Jul 28 19:58:56.538201 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.538150    2554 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 28 19:58:56.544068 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.543958    2554 manager.go:216] Machine: {Timestamp:2026-07-28 19:58:56.542613574 +0000 UTC m=+0.423149558 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3114449 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec277162d0279e854a6022b073d60aee SystemUUID:ec277162-d027-9e85-4a60-22b073d60aee BootID:33e97877-1b44-44e8-b424-c12d2cf7f47b Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:69:09:58:89:49 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:69:09:58:89:49 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:72:c1:c9:b6:cb:39 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 28 19:58:56.544068 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.544063    2554 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 28 19:58:56.544175 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.544139    2554 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 28 19:58:56.546399 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.546382    2554 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 19:58:56.547269 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.547246    2554 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 28 19:58:56.547419 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.547275    2554 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-54.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 28 19:58:56.548032 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.548022    2554 topology_manager.go:138] "Creating topology manager with none policy"
Jul 28 19:58:56.548064 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.548034    2554 container_manager_linux.go:309] "Creating device plugin manager"
Jul 28 19:58:56.548064 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.548042    2554 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 19:58:56.548064 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.548055    2554 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 19:58:56.548951 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.548941    2554 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 28 19:58:56.550748 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.550737    2554 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 19:58:56.550885 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.550875    2554 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 28 19:58:56.550919 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.550911    2554 kubelet.go:486] "Attempting to sync node with API server"
Jul 28 19:58:56.550946 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.550933    2554 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 28 19:58:56.550976 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.550955    2554 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 28 19:58:56.550976 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.550969    2554 kubelet.go:398] "Adding apiserver pod source"
Jul 28 19:58:56.551028 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.550981    2554 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 28 19:58:56.551535 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.551518    2554 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 19:58:56.551638 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.551540    2554 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 19:58:56.554964 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.554931    2554 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 28 19:58:56.557070 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.557054    2554 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 28 19:58:56.557146 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.557085    2554 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 28 19:58:56.558836 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.558822    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 28 19:58:56.558836 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.558837    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 28 19:58:56.558945 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.558844    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 28 19:58:56.558945 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.558849    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 28 19:58:56.558945 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.558858    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 28 19:58:56.558945 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.558863    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 28 19:58:56.558945 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.558872    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 28 19:58:56.558945 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.558878    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 28 19:58:56.558945 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.558885    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 28 19:58:56.558945 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.558891    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 28 19:58:56.558945 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.558900    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 28 19:58:56.558945 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.558906    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 28 19:58:56.559710 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.559697    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 28 19:58:56.559710 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.559706    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 28 19:58:56.560510 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:58:56.560474    2554 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-54.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 28 19:58:56.560510 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:58:56.560474    2554 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 28 19:58:56.562464 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.562446    2554 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-54.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 28 19:58:56.563027 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.563008    2554 server.go:1268] "Started kubelet"
Jul 28 19:58:56.563131 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.563048    2554 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 28 19:58:56.563131 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.563086    2554 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 28 19:58:56.563237 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.563160    2554 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 28 19:58:56.563827 ip-10-0-139-54 systemd[1]: Started Kubernetes Kubelet.
Jul 28 19:58:56.564965 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.564947    2554 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 28 19:58:56.565937 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.565922    2554 server.go:310] "Adding debug handlers to kubelet server"
Jul 28 19:58:56.568417 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:58:56.568393    2554 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 28 19:58:56.571748 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.571732    2554 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 28 19:58:56.571820 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.571748    2554 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 28 19:58:56.572176 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:58:56.571877    2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-54.ec2.internal\" not found"
Jul 28 19:58:56.572436 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:58:56.571097    2554 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-54.ec2.internal.18c68c75076f9ab1  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-54.ec2.internal,UID:ip-10-0-139-54.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-54.ec2.internal,},FirstTimestamp:2026-07-28 19:58:56.562797233 +0000 UTC m=+0.443333197,LastTimestamp:2026-07-28 19:58:56.562797233 +0000 UTC m=+0.443333197,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-54.ec2.internal,}"
Jul 28 19:58:56.572623 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.572601    2554 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 28 19:58:56.572689 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.572613    2554 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 28 19:58:56.572750 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.572655    2554 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 28 19:58:56.572802 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.572780    2554 factory.go:55] Registering systemd factory
Jul 28 19:58:56.572802 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.572798    2554 factory.go:223] Registration of the systemd container factory successfully
Jul 28 19:58:56.572893 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.572835    2554 reconstruct.go:97] "Volume reconstruction finished"
Jul 28 19:58:56.572893 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.572847    2554 reconciler.go:29] "Reconciler: start to sync state"
Jul 28 19:58:56.573042 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.573025    2554 factory.go:153] Registering CRI-O factory
Jul 28 19:58:56.573115 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.573046    2554 factory.go:223] Registration of the crio container factory successfully
Jul 28 19:58:56.573115 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.573104    2554 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 28 19:58:56.573218 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.573128    2554 factory.go:103] Registering Raw factory
Jul 28 19:58:56.573218 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.573146    2554 manager.go:1196] Started watching for new ooms in manager
Jul 28 19:58:56.573670 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.573654    2554 manager.go:349] Starting recovery of all containers
Jul 28 19:58:56.574670 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:58:56.574639    2554 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-54.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 28 19:58:56.574888 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:58:56.574860    2554 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 28 19:58:56.580060 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.580036    2554 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 28 19:58:56.589306 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.589279    2554 manager.go:354] Recovery completed
Jul 28 19:58:56.595419 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.595404    2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:58:56.597976 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.597942    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-54.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:58:56.598053 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.597992    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-54.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:58:56.598053 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.598007    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-54.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:58:56.598881 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.598861    2554 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 28 19:58:56.598950 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.598882    2554 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 28 19:58:56.598950 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.598902    2554 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 19:58:56.598950 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.598867    2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:58:56.599869 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.599846    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-54.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:58:56.599948 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.599876    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-54.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:58:56.599948 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.599900    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-54.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:58:56.600350 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:58:56.600247    2554 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-54.ec2.internal.18c68c750988611b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-54.ec2.internal,UID:ip-10-0-139-54.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-139-54.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-139-54.ec2.internal,},FirstTimestamp:2026-07-28 19:58:56.597975323 +0000 UTC m=+0.478511288,LastTimestamp:2026-07-28 19:58:56.597975323 +0000 UTC m=+0.478511288,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-54.ec2.internal,}"
Jul 28 19:58:56.601179 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.601168    2554 policy_none.go:49] "None policy: Start"
Jul 28 19:58:56.601219 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.601185    2554 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 28 19:58:56.601219 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.601196    2554 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 28 19:58:56.602187 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.602176    2554 policy_none.go:47] "Start"
Jul 28 19:58:56.609033 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.609014    2554 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-m4fz5"
Jul 28 19:58:56.611182 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:58:56.611114    2554 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-54.ec2.internal.18c68c750988bc00  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-54.ec2.internal,UID:ip-10-0-139-54.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-139-54.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-139-54.ec2.internal,},FirstTimestamp:2026-07-28 19:58:56.597998592 +0000 UTC m=+0.478534558,LastTimestamp:2026-07-28 19:58:56.597998592 +0000 UTC m=+0.478534558,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-54.ec2.internal,}"
Jul 28 19:58:56.614319 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.614301    2554 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-m4fz5"
Jul 28 19:58:56.619942 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:58:56.619858    2554 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-54.ec2.internal.18c68c750988f41e  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-54.ec2.internal,UID:ip-10-0-139-54.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-139-54.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-139-54.ec2.internal,},FirstTimestamp:2026-07-28 19:58:56.598012958 +0000 UTC m=+0.478548924,LastTimestamp:2026-07-28 19:58:56.598012958 +0000 UTC m=+0.478548924,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-54.ec2.internal,}"
Jul 28 19:58:56.644618 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.640124    2554 manager.go:337] "Starting Device Plugin manager"
Jul 28 19:58:56.644618 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:58:56.640154    2554 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 28 19:58:56.644618 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.640167    2554 server.go:87] "Starting device plugin registration server"
Jul 28 19:58:56.644618 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.640429    2554 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 28 19:58:56.644618 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.640443    2554 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 28 19:58:56.644618 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.640571    2554 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 28 19:58:56.644618 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.640652    2554 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 28 19:58:56.644618 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.640661    2554 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 28 19:58:56.644618 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:58:56.641151    2554 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 28 19:58:56.644618 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:58:56.641201    2554 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-54.ec2.internal\" not found"
Jul 28 19:58:56.673765 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.673732    2554 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 28 19:58:56.673886 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.673777    2554 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 28 19:58:56.673886 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.673805    2554 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 28 19:58:56.673886 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:58:56.673839    2554 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 28 19:58:56.676371 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.676346    2554 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 19:58:56.741359 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.741329    2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:58:56.742339 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.742316    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-54.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:58:56.742459 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.742351    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-54.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:58:56.742459 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.742362    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-54.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:58:56.742459 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.742385    2554 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-54.ec2.internal"
Jul 28 19:58:56.752086 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.751904    2554 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-54.ec2.internal"
Jul 28 19:58:56.752086 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:58:56.752043    2554 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-54.ec2.internal\": node \"ip-10-0-139-54.ec2.internal\" not found"
Jul 28 19:58:56.774252 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.774225    2554 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-54.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-54.ec2.internal"]
Jul 28 19:58:56.774362 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.774293    2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:58:56.774598 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:58:56.774583    2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-54.ec2.internal\" not found"
Jul 28 19:58:56.775135 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.775118    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-54.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:58:56.775213 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.775145    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-54.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:58:56.775213 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.775157    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-54.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:58:56.776430 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.776418    2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:58:56.776572 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.776558    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-54.ec2.internal"
Jul 28 19:58:56.776617 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.776586    2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:58:56.777184 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.777169    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-54.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:58:56.777262 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.777191    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-54.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:58:56.777262 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.777204    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-54.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:58:56.777348 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.777203    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-54.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:58:56.777348 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.777328    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-54.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:58:56.777348 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.777345    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-54.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:58:56.778865 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.778849    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-54.ec2.internal"
Jul 28 19:58:56.778954 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.778880    2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:58:56.779510 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.779496    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-54.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:58:56.779597 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.779520    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-54.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:58:56.779597 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.779532    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-54.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:58:56.799082 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:58:56.799064    2554 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-54.ec2.internal\" not found" node="ip-10-0-139-54.ec2.internal"
Jul 28 19:58:56.802362 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:58:56.802346    2554 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-54.ec2.internal\" not found" node="ip-10-0-139-54.ec2.internal"
Jul 28 19:58:56.874358 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.874325    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ff4439bcade5003e5697cd6a35723c4e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-54.ec2.internal\" (UID: \"ff4439bcade5003e5697cd6a35723c4e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-54.ec2.internal"
Jul 28 19:58:56.874358 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.874354    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ff4439bcade5003e5697cd6a35723c4e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-54.ec2.internal\" (UID: \"ff4439bcade5003e5697cd6a35723c4e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-54.ec2.internal"
Jul 28 19:58:56.874571 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.874372    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/74845271ca622f52b49ec9ab9ec8f7b9-config\") pod \"kube-apiserver-proxy-ip-10-0-139-54.ec2.internal\" (UID: \"74845271ca622f52b49ec9ab9ec8f7b9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-54.ec2.internal"
Jul 28 19:58:56.875362 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:58:56.875348    2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-54.ec2.internal\" not found"
Jul 28 19:58:56.974815 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.974785    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/74845271ca622f52b49ec9ab9ec8f7b9-config\") pod \"kube-apiserver-proxy-ip-10-0-139-54.ec2.internal\" (UID: \"74845271ca622f52b49ec9ab9ec8f7b9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-54.ec2.internal"
Jul 28 19:58:56.974927 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.974818    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ff4439bcade5003e5697cd6a35723c4e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-54.ec2.internal\" (UID: \"ff4439bcade5003e5697cd6a35723c4e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-54.ec2.internal"
Jul 28 19:58:56.974927 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.974837    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ff4439bcade5003e5697cd6a35723c4e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-54.ec2.internal\" (UID: \"ff4439bcade5003e5697cd6a35723c4e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-54.ec2.internal"
Jul 28 19:58:56.974927 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.974891    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/74845271ca622f52b49ec9ab9ec8f7b9-config\") pod \"kube-apiserver-proxy-ip-10-0-139-54.ec2.internal\" (UID: \"74845271ca622f52b49ec9ab9ec8f7b9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-54.ec2.internal"
Jul 28 19:58:56.975016 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.974924    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ff4439bcade5003e5697cd6a35723c4e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-54.ec2.internal\" (UID: \"ff4439bcade5003e5697cd6a35723c4e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-54.ec2.internal"
Jul 28 19:58:56.975016 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:56.974903    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ff4439bcade5003e5697cd6a35723c4e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-54.ec2.internal\" (UID: \"ff4439bcade5003e5697cd6a35723c4e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-54.ec2.internal"
Jul 28 19:58:56.975842 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:58:56.975826    2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-54.ec2.internal\" not found"
Jul 28 19:58:57.076723 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:58:57.076654    2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-54.ec2.internal\" not found"
Jul 28 19:58:57.101852 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:57.101826    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-54.ec2.internal"
Jul 28 19:58:57.106336 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:57.105890    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-54.ec2.internal"
Jul 28 19:58:57.177587 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:58:57.177530    2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-54.ec2.internal\" not found"
Jul 28 19:58:58.375496 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:58:58.375466    2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-54.ec2.internal\" not found"
Jul 28 19:58:58.470474 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.470407    2554 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 19:58:58.570089 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.570041    2554 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-54.ec2.internal"
Jul 28 19:58:58.582200 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.582175    2554 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 19:58:58.584002 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.583988    2554 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-54.ec2.internal"
Jul 28 19:58:58.595323 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.595301    2554 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 19:58:58.613494 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.613464    2554 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 28 19:58:58.613634 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.613614    2554 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 28 19:58:58.613672 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.613640    2554 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 28 19:58:58.629035 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.629010    2554 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 19:58:58.649217 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.649193    2554 apiserver.go:52] "Watching apiserver"
Jul 28 19:58:58.661500 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.661475    2554 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 28 19:58:58.662919 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.662897    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-qrztd","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-54.ec2.internal","openshift-multus/multus-additional-cni-plugins-7p48k","openshift-network-diagnostics/network-check-target-v2g2z","openshift-network-operator/iptables-alerter-9qlvt","openshift-ovn-kubernetes/ovnkube-node-qxw76","kube-system/global-pull-secret-syncer-rv7fr","kube-system/konnectivity-agent-xs5fh","kube-system/kube-apiserver-proxy-ip-10-0-139-54.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w779p","openshift-cluster-node-tuning-operator/tuned-9lw2g","openshift-dns/node-resolver-42pjq","openshift-multus/multus-xxgzp","openshift-multus/network-metrics-daemon-xmd9l"]
Jul 28 19:58:58.664358 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.664342    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xs5fh"
Jul 28 19:58:58.667093 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.667056    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-42pjq"
Jul 28 19:58:58.667510 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.667487    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-s9j25\""
Jul 28 19:58:58.667628 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.667588    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 28 19:58:58.667693 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.667678    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7p48k"
Jul 28 19:58:58.668600 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.668580    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 28 19:58:58.668943 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.668927    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v2g2z"
Jul 28 19:58:58.669012 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:58:58.668991    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v2g2z" podUID="2f157e54-c70e-46b1-a41b-d60279cd7189"
Jul 28 19:58:58.669372 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.669350    2554 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 28 19:58:58.669577 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.669561    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-lzwf8\""
Jul 28 19:58:58.669925 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.669909    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 28 19:58:58.669989 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.669950    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 28 19:58:58.670038 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.670016    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rv7fr"
Jul 28 19:58:58.670111 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:58:58.670087    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rv7fr" podUID="9cc5727b-a071-4793-a0b1-b92f6287080a"
Jul 28 19:58:58.670337 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.670324    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 28 19:58:58.670511 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.670498    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 28 19:58:58.670921 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.670906    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 28 19:58:58.670969 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.670945    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 28 19:58:58.670969 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.670954    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-nnsqg\""
Jul 28 19:58:58.671392 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.671380    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 28 19:58:58.671436 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.671420    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.672532 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.672518    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qrztd"
Jul 28 19:58:58.673702 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.673685    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9qlvt"
Jul 28 19:58:58.674692 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.674675    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 28 19:58:58.674792 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.674698    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 28 19:58:58.674792 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.674719    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 28 19:58:58.674792 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.674720    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 28 19:58:58.674792 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.674682    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 28 19:58:58.674792 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.674764    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 28 19:58:58.675240 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.675226    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w779p"
Jul 28 19:58:58.675385 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.675363    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ea4abf49-55cf-4351-a952-7b7a83e803d9-cnibin\") pod \"multus-additional-cni-plugins-7p48k\" (UID: \"ea4abf49-55cf-4351-a952-7b7a83e803d9\") " pod="openshift-multus/multus-additional-cni-plugins-7p48k"
Jul 28 19:58:58.675510 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.675397    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f8bb4b77-7948-42bb-9433-d048c7205a61-etc-openvswitch\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.675510 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.675422    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/acc81073-b397-4ed9-8e21-5826814e303e-agent-certs\") pod \"konnectivity-agent-xs5fh\" (UID: \"acc81073-b397-4ed9-8e21-5826814e303e\") " pod="kube-system/konnectivity-agent-xs5fh"
Jul 28 19:58:58.675510 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.675463    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ea4abf49-55cf-4351-a952-7b7a83e803d9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7p48k\" (UID: \"ea4abf49-55cf-4351-a952-7b7a83e803d9\") " pod="openshift-multus/multus-additional-cni-plugins-7p48k"
Jul 28 19:58:58.675510 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.675488    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f8bb4b77-7948-42bb-9433-d048c7205a61-run-openvswitch\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.675743 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.675510    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f8bb4b77-7948-42bb-9433-d048c7205a61-host-cni-netd\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.675743 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.675541    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f8bb4b77-7948-42bb-9433-d048c7205a61-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.675743 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.675571    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c254f770-a33c-4a75-bd75-9bdc3e4e12ed-hosts-file\") pod \"node-resolver-42pjq\" (UID: \"c254f770-a33c-4a75-bd75-9bdc3e4e12ed\") " pod="openshift-dns/node-resolver-42pjq"
Jul 28 19:58:58.675743 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.675586    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c254f770-a33c-4a75-bd75-9bdc3e4e12ed-tmp-dir\") pod \"node-resolver-42pjq\" (UID: \"c254f770-a33c-4a75-bd75-9bdc3e4e12ed\") " pod="openshift-dns/node-resolver-42pjq"
Jul 28 19:58:58.675743 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.675600    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-cbl52\""
Jul 28 19:58:58.675743 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.675602    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ea4abf49-55cf-4351-a952-7b7a83e803d9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7p48k\" (UID: \"ea4abf49-55cf-4351-a952-7b7a83e803d9\") " pod="openshift-multus/multus-additional-cni-plugins-7p48k"
Jul 28 19:58:58.675743 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.675647    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f8bb4b77-7948-42bb-9433-d048c7205a61-host-kubelet\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.675743 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.675655    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 28 19:58:58.675743 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.675664    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f8bb4b77-7948-42bb-9433-d048c7205a61-run-systemd\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.675743 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.675667    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 28 19:58:58.675743 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.675681    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f8bb4b77-7948-42bb-9433-d048c7205a61-env-overrides\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.675743 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.675699    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-l7nw7\""
Jul 28 19:58:58.675743 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.675704    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f8bb4b77-7948-42bb-9433-d048c7205a61-host-cni-bin\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.675743 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.675727    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f8bb4b77-7948-42bb-9433-d048c7205a61-host-run-ovn-kubernetes\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.675743 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.675753    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 28 19:58:58.675743 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.675759    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/acc81073-b397-4ed9-8e21-5826814e303e-konnectivity-ca\") pod \"konnectivity-agent-xs5fh\" (UID: \"acc81073-b397-4ed9-8e21-5826814e303e\") " pod="kube-system/konnectivity-agent-xs5fh"
Jul 28 19:58:58.676383 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.675789    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcjrs\" (UniqueName: \"kubernetes.io/projected/2f157e54-c70e-46b1-a41b-d60279cd7189-kube-api-access-qcjrs\") pod \"network-check-target-v2g2z\" (UID: \"2f157e54-c70e-46b1-a41b-d60279cd7189\") " pod="openshift-network-diagnostics/network-check-target-v2g2z"
Jul 28 19:58:58.676383 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.675815    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f8bb4b77-7948-42bb-9433-d048c7205a61-log-socket\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.676383 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.675837    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drqn5\" (UniqueName: \"kubernetes.io/projected/f8bb4b77-7948-42bb-9433-d048c7205a61-kube-api-access-drqn5\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.676383 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.675855    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xsjp\" (UniqueName: \"kubernetes.io/projected/c254f770-a33c-4a75-bd75-9bdc3e4e12ed-kube-api-access-6xsjp\") pod \"node-resolver-42pjq\" (UID: \"c254f770-a33c-4a75-bd75-9bdc3e4e12ed\") " pod="openshift-dns/node-resolver-42pjq"
Jul 28 19:58:58.676383 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.675869    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ea4abf49-55cf-4351-a952-7b7a83e803d9-os-release\") pod \"multus-additional-cni-plugins-7p48k\" (UID: \"ea4abf49-55cf-4351-a952-7b7a83e803d9\") " pod="openshift-multus/multus-additional-cni-plugins-7p48k"
Jul 28 19:58:58.676383 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.675884    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9cc5727b-a071-4793-a0b1-b92f6287080a-original-pull-secret\") pod \"global-pull-secret-syncer-rv7fr\" (UID: \"9cc5727b-a071-4793-a0b1-b92f6287080a\") " pod="kube-system/global-pull-secret-syncer-rv7fr"
Jul 28 19:58:58.676383 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.675899    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f8bb4b77-7948-42bb-9433-d048c7205a61-node-log\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.676383 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.675914    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f8bb4b77-7948-42bb-9433-d048c7205a61-ovnkube-config\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.676383 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.675928    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f8bb4b77-7948-42bb-9433-d048c7205a61-ovn-node-metrics-cert\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.676383 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.675943    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ea4abf49-55cf-4351-a952-7b7a83e803d9-cni-binary-copy\") pod \"multus-additional-cni-plugins-7p48k\" (UID: \"ea4abf49-55cf-4351-a952-7b7a83e803d9\") " pod="openshift-multus/multus-additional-cni-plugins-7p48k"
Jul 28 19:58:58.676383 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.675958    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9cc5727b-a071-4793-a0b1-b92f6287080a-kubelet-config\") pod \"global-pull-secret-syncer-rv7fr\" (UID: \"9cc5727b-a071-4793-a0b1-b92f6287080a\") " pod="kube-system/global-pull-secret-syncer-rv7fr"
Jul 28 19:58:58.676383 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.676004    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f8bb4b77-7948-42bb-9433-d048c7205a61-host-run-netns\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.676383 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.676026    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f8bb4b77-7948-42bb-9433-d048c7205a61-run-ovn\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.676383 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.676041    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ea4abf49-55cf-4351-a952-7b7a83e803d9-system-cni-dir\") pod \"multus-additional-cni-plugins-7p48k\" (UID: \"ea4abf49-55cf-4351-a952-7b7a83e803d9\") " pod="openshift-multus/multus-additional-cni-plugins-7p48k"
Jul 28 19:58:58.676383 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.676054    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ea4abf49-55cf-4351-a952-7b7a83e803d9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7p48k\" (UID: \"ea4abf49-55cf-4351-a952-7b7a83e803d9\") " pod="openshift-multus/multus-additional-cni-plugins-7p48k"
Jul 28 19:58:58.676383 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.676065    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 28 19:58:58.676856 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.676090    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6k2kw\" (UniqueName: \"kubernetes.io/projected/ea4abf49-55cf-4351-a952-7b7a83e803d9-kube-api-access-6k2kw\") pod \"multus-additional-cni-plugins-7p48k\" (UID: \"ea4abf49-55cf-4351-a952-7b7a83e803d9\") " pod="openshift-multus/multus-additional-cni-plugins-7p48k"
Jul 28 19:58:58.676856 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.676121    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9cc5727b-a071-4793-a0b1-b92f6287080a-dbus\") pod \"global-pull-secret-syncer-rv7fr\" (UID: \"9cc5727b-a071-4793-a0b1-b92f6287080a\") " pod="kube-system/global-pull-secret-syncer-rv7fr"
Jul 28 19:58:58.676856 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.676156    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f8bb4b77-7948-42bb-9433-d048c7205a61-systemd-units\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.676856 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.676191    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 28 19:58:58.676856 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.676194    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f8bb4b77-7948-42bb-9433-d048c7205a61-host-slash\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.676856 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.676220    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f8bb4b77-7948-42bb-9433-d048c7205a61-var-lib-openvswitch\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.676856 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.676237    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f8bb4b77-7948-42bb-9433-d048c7205a61-ovnkube-script-lib\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.676856 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.676636    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9lw2g"
Jul 28 19:58:58.676856 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.676854    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 28 19:58:58.677205 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.676862    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-h7694\""
Jul 28 19:58:58.678264 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.678250    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-k6699\""
Jul 28 19:58:58.678359 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.678344    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-xxgzp"
Jul 28 19:58:58.678420 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.678349    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 28 19:58:58.679281 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.679265    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 28 19:58:58.679516 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.679492    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 28 19:58:58.679598 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.679582    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-hlsrq\""
Jul 28 19:58:58.679652 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.679495    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 28 19:58:58.679702 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.679652    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 28 19:58:58.679819 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.679805    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xmd9l"
Jul 28 19:58:58.679884 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:58:58.679868    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xmd9l" podUID="752faedf-0642-47aa-88bb-72f0ae961ebb"
Jul 28 19:58:58.680957 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.680940    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-fcfqc\""
Jul 28 19:58:58.681025 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.680940    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 28 19:58:58.682795 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.682778    2554 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 19:58:58.704420 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.704396    2554 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-wpmlq"
Jul 28 19:58:58.712402 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.712377    2554 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-wpmlq"
Jul 28 19:58:58.716342 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.716242    2554 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-27 19:53:57 +0000 UTC" deadline="2028-04-30 03:59:03.335302006 +0000 UTC"
Jul 28 19:58:58.716579 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.716564    2554 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15392h0m4.618744371s"
Jul 28 19:58:58.771020 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.770990    2554 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 28 19:58:58.776885 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.776863    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f8bb4b77-7948-42bb-9433-d048c7205a61-ovnkube-script-lib\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.776985 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.776891    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ea4abf49-55cf-4351-a952-7b7a83e803d9-cnibin\") pod \"multus-additional-cni-plugins-7p48k\" (UID: \"ea4abf49-55cf-4351-a952-7b7a83e803d9\") " pod="openshift-multus/multus-additional-cni-plugins-7p48k"
Jul 28 19:58:58.776985 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.776910    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/acc81073-b397-4ed9-8e21-5826814e303e-agent-certs\") pod \"konnectivity-agent-xs5fh\" (UID: \"acc81073-b397-4ed9-8e21-5826814e303e\") " pod="kube-system/konnectivity-agent-xs5fh"
Jul 28 19:58:58.776985 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.776927    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/38cf5bd2-15db-49bc-b41d-b0df3ea06cb1-serviceca\") pod \"node-ca-qrztd\" (UID: \"38cf5bd2-15db-49bc-b41d-b0df3ea06cb1\") " pod="openshift-image-registry/node-ca-qrztd"
Jul 28 19:58:58.776985 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.776970    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/746a745b-0cf6-4b92-8173-4a8932ec5dff-etc-selinux\") pod \"aws-ebs-csi-driver-node-w779p\" (UID: \"746a745b-0cf6-4b92-8173-4a8932ec5dff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w779p"
Jul 28 19:58:58.777122 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.776993    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ea4abf49-55cf-4351-a952-7b7a83e803d9-cnibin\") pod \"multus-additional-cni-plugins-7p48k\" (UID: \"ea4abf49-55cf-4351-a952-7b7a83e803d9\") " pod="openshift-multus/multus-additional-cni-plugins-7p48k"
Jul 28 19:58:58.777122 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.777054    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/95c98833-0ec7-45eb-8986-14d40c9986e8-etc-tuned\") pod \"tuned-9lw2g\" (UID: \"95c98833-0ec7-45eb-8986-14d40c9986e8\") " pod="openshift-cluster-node-tuning-operator/tuned-9lw2g"
Jul 28 19:58:58.777122 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.777110    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6485753f-f774-4aa4-aaec-130c7b7ba868-host-var-lib-kubelet\") pod \"multus-xxgzp\" (UID: \"6485753f-f774-4aa4-aaec-130c7b7ba868\") " pod="openshift-multus/multus-xxgzp"
Jul 28 19:58:58.777206 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.777153    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f8bb4b77-7948-42bb-9433-d048c7205a61-run-openvswitch\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.777243 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.777204    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f8bb4b77-7948-42bb-9433-d048c7205a61-host-cni-netd\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.777243 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.777223    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f8bb4b77-7948-42bb-9433-d048c7205a61-run-openvswitch\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.777310 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.777239    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f8bb4b77-7948-42bb-9433-d048c7205a61-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.777310 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.777284    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f8bb4b77-7948-42bb-9433-d048c7205a61-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.777310 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.777293    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c254f770-a33c-4a75-bd75-9bdc3e4e12ed-tmp-dir\") pod \"node-resolver-42pjq\" (UID: \"c254f770-a33c-4a75-bd75-9bdc3e4e12ed\") " pod="openshift-dns/node-resolver-42pjq"
Jul 28 19:58:58.777310 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.777300    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f8bb4b77-7948-42bb-9433-d048c7205a61-host-cni-netd\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.777474 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.777320    2554 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 28 19:58:58.777474 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.777335    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/746a745b-0cf6-4b92-8173-4a8932ec5dff-device-dir\") pod \"aws-ebs-csi-driver-node-w779p\" (UID: \"746a745b-0cf6-4b92-8173-4a8932ec5dff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w779p"
Jul 28 19:58:58.777474 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.777362    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ea4abf49-55cf-4351-a952-7b7a83e803d9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7p48k\" (UID: \"ea4abf49-55cf-4351-a952-7b7a83e803d9\") " pod="openshift-multus/multus-additional-cni-plugins-7p48k"
Jul 28 19:58:58.777474 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.777414    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f8bb4b77-7948-42bb-9433-d048c7205a61-run-systemd\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.777474 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.777457    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/746a745b-0cf6-4b92-8173-4a8932ec5dff-socket-dir\") pod \"aws-ebs-csi-driver-node-w779p\" (UID: \"746a745b-0cf6-4b92-8173-4a8932ec5dff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w779p"
Jul 28 19:58:58.777760 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.777473    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f8bb4b77-7948-42bb-9433-d048c7205a61-run-systemd\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.777760 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.777482    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/95c98833-0ec7-45eb-8986-14d40c9986e8-run\") pod \"tuned-9lw2g\" (UID: \"95c98833-0ec7-45eb-8986-14d40c9986e8\") " pod="openshift-cluster-node-tuning-operator/tuned-9lw2g"
Jul 28 19:58:58.777760 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.777523    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6485753f-f774-4aa4-aaec-130c7b7ba868-host-run-netns\") pod \"multus-xxgzp\" (UID: \"6485753f-f774-4aa4-aaec-130c7b7ba868\") " pod="openshift-multus/multus-xxgzp"
Jul 28 19:58:58.777760 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.777528    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c254f770-a33c-4a75-bd75-9bdc3e4e12ed-tmp-dir\") pod \"node-resolver-42pjq\" (UID: \"c254f770-a33c-4a75-bd75-9bdc3e4e12ed\") " pod="openshift-dns/node-resolver-42pjq"
Jul 28 19:58:58.777760 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.777557    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f8bb4b77-7948-42bb-9433-d048c7205a61-host-cni-bin\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.777760 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.777583    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f8bb4b77-7948-42bb-9433-d048c7205a61-host-run-ovn-kubernetes\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.777760 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.777611    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/746a745b-0cf6-4b92-8173-4a8932ec5dff-sys-fs\") pod \"aws-ebs-csi-driver-node-w779p\" (UID: \"746a745b-0cf6-4b92-8173-4a8932ec5dff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w779p"
Jul 28 19:58:58.777760 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.777616    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f8bb4b77-7948-42bb-9433-d048c7205a61-host-cni-bin\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.777760 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.777636    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6485753f-f774-4aa4-aaec-130c7b7ba868-os-release\") pod \"multus-xxgzp\" (UID: \"6485753f-f774-4aa4-aaec-130c7b7ba868\") " pod="openshift-multus/multus-xxgzp"
Jul 28 19:58:58.777760 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.777645    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f8bb4b77-7948-42bb-9433-d048c7205a61-host-run-ovn-kubernetes\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.777760 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.777659    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6485753f-f774-4aa4-aaec-130c7b7ba868-etc-kubernetes\") pod \"multus-xxgzp\" (UID: \"6485753f-f774-4aa4-aaec-130c7b7ba868\") " pod="openshift-multus/multus-xxgzp"
Jul 28 19:58:58.777760 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.777693    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xsjp\" (UniqueName: \"kubernetes.io/projected/c254f770-a33c-4a75-bd75-9bdc3e4e12ed-kube-api-access-6xsjp\") pod \"node-resolver-42pjq\" (UID: \"c254f770-a33c-4a75-bd75-9bdc3e4e12ed\") " pod="openshift-dns/node-resolver-42pjq"
Jul 28 19:58:58.777760 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.777721    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6485753f-f774-4aa4-aaec-130c7b7ba868-multus-socket-dir-parent\") pod \"multus-xxgzp\" (UID: \"6485753f-f774-4aa4-aaec-130c7b7ba868\") " pod="openshift-multus/multus-xxgzp"
Jul 28 19:58:58.778385 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.777748    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6485753f-f774-4aa4-aaec-130c7b7ba868-host-run-k8s-cni-cncf-io\") pod \"multus-xxgzp\" (UID: \"6485753f-f774-4aa4-aaec-130c7b7ba868\") " pod="openshift-multus/multus-xxgzp"
Jul 28 19:58:58.778385 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.777804    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/03099009-562b-4d4b-9db8-95ec824e4b1c-host-slash\") pod \"iptables-alerter-9qlvt\" (UID: \"03099009-562b-4d4b-9db8-95ec824e4b1c\") " pod="openshift-network-operator/iptables-alerter-9qlvt"
Jul 28 19:58:58.778385 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.777832    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ea4abf49-55cf-4351-a952-7b7a83e803d9-os-release\") pod \"multus-additional-cni-plugins-7p48k\" (UID: \"ea4abf49-55cf-4351-a952-7b7a83e803d9\") " pod="openshift-multus/multus-additional-cni-plugins-7p48k"
Jul 28 19:58:58.778385 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.777870    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9cc5727b-a071-4793-a0b1-b92f6287080a-original-pull-secret\") pod \"global-pull-secret-syncer-rv7fr\" (UID: \"9cc5727b-a071-4793-a0b1-b92f6287080a\") " pod="kube-system/global-pull-secret-syncer-rv7fr"
Jul 28 19:58:58.778385 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.777897    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f8bb4b77-7948-42bb-9433-d048c7205a61-node-log\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.778385 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.777912    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ea4abf49-55cf-4351-a952-7b7a83e803d9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7p48k\" (UID: \"ea4abf49-55cf-4351-a952-7b7a83e803d9\") " pod="openshift-multus/multus-additional-cni-plugins-7p48k"
Jul 28 19:58:58.778385 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.777925    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ea4abf49-55cf-4351-a952-7b7a83e803d9-os-release\") pod \"multus-additional-cni-plugins-7p48k\" (UID: \"ea4abf49-55cf-4351-a952-7b7a83e803d9\") " pod="openshift-multus/multus-additional-cni-plugins-7p48k"
Jul 28 19:58:58.778385 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.777929    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f8bb4b77-7948-42bb-9433-d048c7205a61-ovnkube-config\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.778385 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.777991    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/746a745b-0cf6-4b92-8173-4a8932ec5dff-registration-dir\") pod \"aws-ebs-csi-driver-node-w779p\" (UID: \"746a745b-0cf6-4b92-8173-4a8932ec5dff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w779p"
Jul 28 19:58:58.778385 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.777985    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f8bb4b77-7948-42bb-9433-d048c7205a61-node-log\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.778385 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:58:58.778023    2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:58:58.778385 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.778020    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/95c98833-0ec7-45eb-8986-14d40c9986e8-etc-sysctl-conf\") pod \"tuned-9lw2g\" (UID: \"95c98833-0ec7-45eb-8986-14d40c9986e8\") " pod="openshift-cluster-node-tuning-operator/tuned-9lw2g"
Jul 28 19:58:58.778385 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.778053    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/752faedf-0642-47aa-88bb-72f0ae961ebb-metrics-certs\") pod \"network-metrics-daemon-xmd9l\" (UID: \"752faedf-0642-47aa-88bb-72f0ae961ebb\") " pod="openshift-multus/network-metrics-daemon-xmd9l"
Jul 28 19:58:58.778385 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.778096    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9cc5727b-a071-4793-a0b1-b92f6287080a-kubelet-config\") pod \"global-pull-secret-syncer-rv7fr\" (UID: \"9cc5727b-a071-4793-a0b1-b92f6287080a\") " pod="kube-system/global-pull-secret-syncer-rv7fr"
Jul 28 19:58:58.778385 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:58:58.778132    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9cc5727b-a071-4793-a0b1-b92f6287080a-original-pull-secret podName:9cc5727b-a071-4793-a0b1-b92f6287080a nodeName:}" failed. No retries permitted until 2026-07-28 19:58:59.27809438 +0000 UTC m=+2.061105819 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9cc5727b-a071-4793-a0b1-b92f6287080a-original-pull-secret") pod "global-pull-secret-syncer-rv7fr" (UID: "9cc5727b-a071-4793-a0b1-b92f6287080a") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:58:58.778385 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.778161    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9cc5727b-a071-4793-a0b1-b92f6287080a-kubelet-config\") pod \"global-pull-secret-syncer-rv7fr\" (UID: \"9cc5727b-a071-4793-a0b1-b92f6287080a\") " pod="kube-system/global-pull-secret-syncer-rv7fr"
Jul 28 19:58:58.779096 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.778205    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f8bb4b77-7948-42bb-9433-d048c7205a61-run-ovn\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.779096 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.778221    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/38cf5bd2-15db-49bc-b41d-b0df3ea06cb1-host\") pod \"node-ca-qrztd\" (UID: \"38cf5bd2-15db-49bc-b41d-b0df3ea06cb1\") " pod="openshift-image-registry/node-ca-qrztd"
Jul 28 19:58:58.779096 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.778234    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/95c98833-0ec7-45eb-8986-14d40c9986e8-sys\") pod \"tuned-9lw2g\" (UID: \"95c98833-0ec7-45eb-8986-14d40c9986e8\") " pod="openshift-cluster-node-tuning-operator/tuned-9lw2g"
Jul 28 19:58:58.779096 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.778256    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/95c98833-0ec7-45eb-8986-14d40c9986e8-lib-modules\") pod \"tuned-9lw2g\" (UID: \"95c98833-0ec7-45eb-8986-14d40c9986e8\") " pod="openshift-cluster-node-tuning-operator/tuned-9lw2g"
Jul 28 19:58:58.779096 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.778282    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6485753f-f774-4aa4-aaec-130c7b7ba868-system-cni-dir\") pod \"multus-xxgzp\" (UID: \"6485753f-f774-4aa4-aaec-130c7b7ba868\") " pod="openshift-multus/multus-xxgzp"
Jul 28 19:58:58.779096 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.778282    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f8bb4b77-7948-42bb-9433-d048c7205a61-run-ovn\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.779096 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.778316    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gx92\" (UniqueName: \"kubernetes.io/projected/752faedf-0642-47aa-88bb-72f0ae961ebb-kube-api-access-5gx92\") pod \"network-metrics-daemon-xmd9l\" (UID: \"752faedf-0642-47aa-88bb-72f0ae961ebb\") " pod="openshift-multus/network-metrics-daemon-xmd9l"
Jul 28 19:58:58.779096 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.778334    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/95c98833-0ec7-45eb-8986-14d40c9986e8-etc-systemd\") pod \"tuned-9lw2g\" (UID: \"95c98833-0ec7-45eb-8986-14d40c9986e8\") " pod="openshift-cluster-node-tuning-operator/tuned-9lw2g"
Jul 28 19:58:58.779096 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.778351    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ea4abf49-55cf-4351-a952-7b7a83e803d9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7p48k\" (UID: \"ea4abf49-55cf-4351-a952-7b7a83e803d9\") " pod="openshift-multus/multus-additional-cni-plugins-7p48k"
Jul 28 19:58:58.779096 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.778386    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f8bb4b77-7948-42bb-9433-d048c7205a61-host-slash\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.779096 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.778414    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6485753f-f774-4aa4-aaec-130c7b7ba868-multus-cni-dir\") pod \"multus-xxgzp\" (UID: \"6485753f-f774-4aa4-aaec-130c7b7ba868\") " pod="openshift-multus/multus-xxgzp"
Jul 28 19:58:58.779096 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.778435    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6485753f-f774-4aa4-aaec-130c7b7ba868-host-run-multus-certs\") pod \"multus-xxgzp\" (UID: \"6485753f-f774-4aa4-aaec-130c7b7ba868\") " pod="openshift-multus/multus-xxgzp"
Jul 28 19:58:58.779096 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.778440    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f8bb4b77-7948-42bb-9433-d048c7205a61-host-slash\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.779096 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.778452    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f8bb4b77-7948-42bb-9433-d048c7205a61-etc-openvswitch\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.779096 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.778473    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6485753f-f774-4aa4-aaec-130c7b7ba868-host-var-lib-cni-multus\") pod \"multus-xxgzp\" (UID: \"6485753f-f774-4aa4-aaec-130c7b7ba868\") " pod="openshift-multus/multus-xxgzp"
Jul 28 19:58:58.779096 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.778502    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ea4abf49-55cf-4351-a952-7b7a83e803d9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7p48k\" (UID: \"ea4abf49-55cf-4351-a952-7b7a83e803d9\") " pod="openshift-multus/multus-additional-cni-plugins-7p48k"
Jul 28 19:58:58.779096 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.778522    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f8bb4b77-7948-42bb-9433-d048c7205a61-etc-openvswitch\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.779872 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.778529    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c254f770-a33c-4a75-bd75-9bdc3e4e12ed-hosts-file\") pod \"node-resolver-42pjq\" (UID: \"c254f770-a33c-4a75-bd75-9bdc3e4e12ed\") " pod="openshift-dns/node-resolver-42pjq"
Jul 28 19:58:58.779872 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.778559    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7lx9\" (UniqueName: \"kubernetes.io/projected/38cf5bd2-15db-49bc-b41d-b0df3ea06cb1-kube-api-access-p7lx9\") pod \"node-ca-qrztd\" (UID: \"38cf5bd2-15db-49bc-b41d-b0df3ea06cb1\") " pod="openshift-image-registry/node-ca-qrztd"
Jul 28 19:58:58.779872 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.778568    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f8bb4b77-7948-42bb-9433-d048c7205a61-ovnkube-config\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.779872 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.778582    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/746a745b-0cf6-4b92-8173-4a8932ec5dff-kubelet-dir\") pod \"aws-ebs-csi-driver-node-w779p\" (UID: \"746a745b-0cf6-4b92-8173-4a8932ec5dff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w779p"
Jul 28 19:58:58.779872 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.778593    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ea4abf49-55cf-4351-a952-7b7a83e803d9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7p48k\" (UID: \"ea4abf49-55cf-4351-a952-7b7a83e803d9\") " pod="openshift-multus/multus-additional-cni-plugins-7p48k"
Jul 28 19:58:58.779872 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.778573    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c254f770-a33c-4a75-bd75-9bdc3e4e12ed-hosts-file\") pod \"node-resolver-42pjq\" (UID: \"c254f770-a33c-4a75-bd75-9bdc3e4e12ed\") " pod="openshift-dns/node-resolver-42pjq"
Jul 28 19:58:58.779872 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.778605    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6485753f-f774-4aa4-aaec-130c7b7ba868-hostroot\") pod \"multus-xxgzp\" (UID: \"6485753f-f774-4aa4-aaec-130c7b7ba868\") " pod="openshift-multus/multus-xxgzp"
Jul 28 19:58:58.779872 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.778627    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f8bb4b77-7948-42bb-9433-d048c7205a61-ovnkube-script-lib\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.779872 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.778672    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/03099009-562b-4d4b-9db8-95ec824e4b1c-iptables-alerter-script\") pod \"iptables-alerter-9qlvt\" (UID: \"03099009-562b-4d4b-9db8-95ec824e4b1c\") " pod="openshift-network-operator/iptables-alerter-9qlvt"
Jul 28 19:58:58.779872 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.778704    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f8bb4b77-7948-42bb-9433-d048c7205a61-host-kubelet\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.779872 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.778728    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f8bb4b77-7948-42bb-9433-d048c7205a61-env-overrides\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.779872 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.778762    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/95c98833-0ec7-45eb-8986-14d40c9986e8-etc-sysctl-d\") pod \"tuned-9lw2g\" (UID: \"95c98833-0ec7-45eb-8986-14d40c9986e8\") " pod="openshift-cluster-node-tuning-operator/tuned-9lw2g"
Jul 28 19:58:58.779872 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.778788    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/95c98833-0ec7-45eb-8986-14d40c9986e8-host\") pod \"tuned-9lw2g\" (UID: \"95c98833-0ec7-45eb-8986-14d40c9986e8\") " pod="openshift-cluster-node-tuning-operator/tuned-9lw2g"
Jul 28 19:58:58.779872 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.778794    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f8bb4b77-7948-42bb-9433-d048c7205a61-host-kubelet\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.779872 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.778809    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6485753f-f774-4aa4-aaec-130c7b7ba868-cni-binary-copy\") pod \"multus-xxgzp\" (UID: \"6485753f-f774-4aa4-aaec-130c7b7ba868\") " pod="openshift-multus/multus-xxgzp"
Jul 28 19:58:58.779872 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.778833    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/acc81073-b397-4ed9-8e21-5826814e303e-konnectivity-ca\") pod \"konnectivity-agent-xs5fh\" (UID: \"acc81073-b397-4ed9-8e21-5826814e303e\") " pod="kube-system/konnectivity-agent-xs5fh"
Jul 28 19:58:58.779872 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.778854    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/95c98833-0ec7-45eb-8986-14d40c9986e8-etc-modprobe-d\") pod \"tuned-9lw2g\" (UID: \"95c98833-0ec7-45eb-8986-14d40c9986e8\") " pod="openshift-cluster-node-tuning-operator/tuned-9lw2g"
Jul 28 19:58:58.780461 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.778875    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/95c98833-0ec7-45eb-8986-14d40c9986e8-etc-sysconfig\") pod \"tuned-9lw2g\" (UID: \"95c98833-0ec7-45eb-8986-14d40c9986e8\") " pod="openshift-cluster-node-tuning-operator/tuned-9lw2g"
Jul 28 19:58:58.780461 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.778896    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95c98833-0ec7-45eb-8986-14d40c9986e8-etc-kubernetes\") pod \"tuned-9lw2g\" (UID: \"95c98833-0ec7-45eb-8986-14d40c9986e8\") " pod="openshift-cluster-node-tuning-operator/tuned-9lw2g"
Jul 28 19:58:58.780461 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.778935    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcjrs\" (UniqueName: \"kubernetes.io/projected/2f157e54-c70e-46b1-a41b-d60279cd7189-kube-api-access-qcjrs\") pod \"network-check-target-v2g2z\" (UID: \"2f157e54-c70e-46b1-a41b-d60279cd7189\") " pod="openshift-network-diagnostics/network-check-target-v2g2z"
Jul 28 19:58:58.780461 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.778965    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f8bb4b77-7948-42bb-9433-d048c7205a61-log-socket\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.780461 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.778987    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drqn5\" (UniqueName: \"kubernetes.io/projected/f8bb4b77-7948-42bb-9433-d048c7205a61-kube-api-access-drqn5\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.780461 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.779015    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6485753f-f774-4aa4-aaec-130c7b7ba868-host-var-lib-cni-bin\") pod \"multus-xxgzp\" (UID: \"6485753f-f774-4aa4-aaec-130c7b7ba868\") " pod="openshift-multus/multus-xxgzp"
Jul 28 19:58:58.780461 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.779017    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ea4abf49-55cf-4351-a952-7b7a83e803d9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7p48k\" (UID: \"ea4abf49-55cf-4351-a952-7b7a83e803d9\") " pod="openshift-multus/multus-additional-cni-plugins-7p48k"
Jul 28 19:58:58.780461 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.779041    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6485753f-f774-4aa4-aaec-130c7b7ba868-multus-conf-dir\") pod \"multus-xxgzp\" (UID: \"6485753f-f774-4aa4-aaec-130c7b7ba868\") " pod="openshift-multus/multus-xxgzp"
Jul 28 19:58:58.780461 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.779081    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dt7v\" (UniqueName: \"kubernetes.io/projected/03099009-562b-4d4b-9db8-95ec824e4b1c-kube-api-access-2dt7v\") pod \"iptables-alerter-9qlvt\" (UID: \"03099009-562b-4d4b-9db8-95ec824e4b1c\") " pod="openshift-network-operator/iptables-alerter-9qlvt"
Jul 28 19:58:58.780461 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.779108    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f8bb4b77-7948-42bb-9433-d048c7205a61-ovn-node-metrics-cert\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.780461 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.779130    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/95c98833-0ec7-45eb-8986-14d40c9986e8-tmp\") pod \"tuned-9lw2g\" (UID: \"95c98833-0ec7-45eb-8986-14d40c9986e8\") " pod="openshift-cluster-node-tuning-operator/tuned-9lw2g"
Jul 28 19:58:58.780461 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.779153    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6485753f-f774-4aa4-aaec-130c7b7ba868-cnibin\") pod \"multus-xxgzp\" (UID: \"6485753f-f774-4aa4-aaec-130c7b7ba868\") " pod="openshift-multus/multus-xxgzp"
Jul 28 19:58:58.780461 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.779163    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f8bb4b77-7948-42bb-9433-d048c7205a61-env-overrides\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.780461 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.779177    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6485753f-f774-4aa4-aaec-130c7b7ba868-multus-daemon-config\") pod \"multus-xxgzp\" (UID: \"6485753f-f774-4aa4-aaec-130c7b7ba868\") " pod="openshift-multus/multus-xxgzp"
Jul 28 19:58:58.780461 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.779234    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f8bb4b77-7948-42bb-9433-d048c7205a61-log-socket\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.780461 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.779278    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ea4abf49-55cf-4351-a952-7b7a83e803d9-cni-binary-copy\") pod \"multus-additional-cni-plugins-7p48k\" (UID: \"ea4abf49-55cf-4351-a952-7b7a83e803d9\") " pod="openshift-multus/multus-additional-cni-plugins-7p48k"
Jul 28 19:58:58.780461 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.779314    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f8bb4b77-7948-42bb-9433-d048c7205a61-host-run-netns\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.781048 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.779407    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/95c98833-0ec7-45eb-8986-14d40c9986e8-var-lib-kubelet\") pod \"tuned-9lw2g\" (UID: \"95c98833-0ec7-45eb-8986-14d40c9986e8\") " pod="openshift-cluster-node-tuning-operator/tuned-9lw2g"
Jul 28 19:58:58.781048 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.779438    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f8bb4b77-7948-42bb-9433-d048c7205a61-host-run-netns\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.781048 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.779467    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkrsp\" (UniqueName: \"kubernetes.io/projected/95c98833-0ec7-45eb-8986-14d40c9986e8-kube-api-access-kkrsp\") pod \"tuned-9lw2g\" (UID: \"95c98833-0ec7-45eb-8986-14d40c9986e8\") " pod="openshift-cluster-node-tuning-operator/tuned-9lw2g"
Jul 28 19:58:58.781048 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.779509    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2wxk\" (UniqueName: \"kubernetes.io/projected/6485753f-f774-4aa4-aaec-130c7b7ba868-kube-api-access-s2wxk\") pod \"multus-xxgzp\" (UID: \"6485753f-f774-4aa4-aaec-130c7b7ba868\") " pod="openshift-multus/multus-xxgzp"
Jul 28 19:58:58.781048 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.779535    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2nbz\" (UniqueName: \"kubernetes.io/projected/746a745b-0cf6-4b92-8173-4a8932ec5dff-kube-api-access-f2nbz\") pod \"aws-ebs-csi-driver-node-w779p\" (UID: \"746a745b-0cf6-4b92-8173-4a8932ec5dff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w779p"
Jul 28 19:58:58.781048 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.779562    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ea4abf49-55cf-4351-a952-7b7a83e803d9-system-cni-dir\") pod \"multus-additional-cni-plugins-7p48k\" (UID: \"ea4abf49-55cf-4351-a952-7b7a83e803d9\") " pod="openshift-multus/multus-additional-cni-plugins-7p48k"
Jul 28 19:58:58.781048 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.779589    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6k2kw\" (UniqueName: \"kubernetes.io/projected/ea4abf49-55cf-4351-a952-7b7a83e803d9-kube-api-access-6k2kw\") pod \"multus-additional-cni-plugins-7p48k\" (UID: \"ea4abf49-55cf-4351-a952-7b7a83e803d9\") " pod="openshift-multus/multus-additional-cni-plugins-7p48k"
Jul 28 19:58:58.781048 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.779614    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9cc5727b-a071-4793-a0b1-b92f6287080a-dbus\") pod \"global-pull-secret-syncer-rv7fr\" (UID: \"9cc5727b-a071-4793-a0b1-b92f6287080a\") " pod="kube-system/global-pull-secret-syncer-rv7fr"
Jul 28 19:58:58.781048 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.779638    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f8bb4b77-7948-42bb-9433-d048c7205a61-systemd-units\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.781048 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.779661    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f8bb4b77-7948-42bb-9433-d048c7205a61-var-lib-openvswitch\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.781048 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.779724    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/acc81073-b397-4ed9-8e21-5826814e303e-konnectivity-ca\") pod \"konnectivity-agent-xs5fh\" (UID: \"acc81073-b397-4ed9-8e21-5826814e303e\") " pod="kube-system/konnectivity-agent-xs5fh"
Jul 28 19:58:58.781048 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.779773    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ea4abf49-55cf-4351-a952-7b7a83e803d9-system-cni-dir\") pod \"multus-additional-cni-plugins-7p48k\" (UID: \"ea4abf49-55cf-4351-a952-7b7a83e803d9\") " pod="openshift-multus/multus-additional-cni-plugins-7p48k"
Jul 28 19:58:58.781048 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.779788    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f8bb4b77-7948-42bb-9433-d048c7205a61-var-lib-openvswitch\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.781048 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.779791    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ea4abf49-55cf-4351-a952-7b7a83e803d9-cni-binary-copy\") pod \"multus-additional-cni-plugins-7p48k\" (UID: \"ea4abf49-55cf-4351-a952-7b7a83e803d9\") " pod="openshift-multus/multus-additional-cni-plugins-7p48k"
Jul 28 19:58:58.781048 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.779799    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9cc5727b-a071-4793-a0b1-b92f6287080a-dbus\") pod \"global-pull-secret-syncer-rv7fr\" (UID: \"9cc5727b-a071-4793-a0b1-b92f6287080a\") " pod="kube-system/global-pull-secret-syncer-rv7fr"
Jul 28 19:58:58.781048 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.779812    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f8bb4b77-7948-42bb-9433-d048c7205a61-systemd-units\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.781738 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.781429    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/acc81073-b397-4ed9-8e21-5826814e303e-agent-certs\") pod \"konnectivity-agent-xs5fh\" (UID: \"acc81073-b397-4ed9-8e21-5826814e303e\") " pod="kube-system/konnectivity-agent-xs5fh"
Jul 28 19:58:58.781738 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.781639    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f8bb4b77-7948-42bb-9433-d048c7205a61-ovn-node-metrics-cert\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.784845 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:58:58.784829    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:58:58.784965 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:58:58.784851    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:58:58.784965 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:58:58.784863    2554 projected.go:196] Error preparing data for projected volume kube-api-access-qcjrs for pod openshift-network-diagnostics/network-check-target-v2g2z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:58:58.784965 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:58:58.784917    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f157e54-c70e-46b1-a41b-d60279cd7189-kube-api-access-qcjrs podName:2f157e54-c70e-46b1-a41b-d60279cd7189 nodeName:}" failed. No retries permitted until 2026-07-28 19:58:59.284901621 +0000 UTC m=+2.067913056 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-qcjrs" (UniqueName: "kubernetes.io/projected/2f157e54-c70e-46b1-a41b-d60279cd7189-kube-api-access-qcjrs") pod "network-check-target-v2g2z" (UID: "2f157e54-c70e-46b1-a41b-d60279cd7189") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:58:58.785483 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.785458    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xsjp\" (UniqueName: \"kubernetes.io/projected/c254f770-a33c-4a75-bd75-9bdc3e4e12ed-kube-api-access-6xsjp\") pod \"node-resolver-42pjq\" (UID: \"c254f770-a33c-4a75-bd75-9bdc3e4e12ed\") " pod="openshift-dns/node-resolver-42pjq"
Jul 28 19:58:58.787196 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.787171    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drqn5\" (UniqueName: \"kubernetes.io/projected/f8bb4b77-7948-42bb-9433-d048c7205a61-kube-api-access-drqn5\") pod \"ovnkube-node-qxw76\" (UID: \"f8bb4b77-7948-42bb-9433-d048c7205a61\") " pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:58.787407 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.787385    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6k2kw\" (UniqueName: \"kubernetes.io/projected/ea4abf49-55cf-4351-a952-7b7a83e803d9-kube-api-access-6k2kw\") pod \"multus-additional-cni-plugins-7p48k\" (UID: \"ea4abf49-55cf-4351-a952-7b7a83e803d9\") " pod="openshift-multus/multus-additional-cni-plugins-7p48k"
Jul 28 19:58:58.855026 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:58.854996    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff4439bcade5003e5697cd6a35723c4e.slice/crio-bbba22234fa1c9b6d825a3bee300390e7623a18d1e5b808106523b0125985d6e WatchSource:0}: Error finding container bbba22234fa1c9b6d825a3bee300390e7623a18d1e5b808106523b0125985d6e: Status 404 returned error can't find the container with id bbba22234fa1c9b6d825a3bee300390e7623a18d1e5b808106523b0125985d6e
Jul 28 19:58:58.855542 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:58.855521    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74845271ca622f52b49ec9ab9ec8f7b9.slice/crio-86ffd49c435be2460d107807187acb6ad2a0cac9e79f852b30379d3bcb00041c WatchSource:0}: Error finding container 86ffd49c435be2460d107807187acb6ad2a0cac9e79f852b30379d3bcb00041c: Status 404 returned error can't find the container with id 86ffd49c435be2460d107807187acb6ad2a0cac9e79f852b30379d3bcb00041c
Jul 28 19:58:58.861920 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.861901    2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 19:58:58.880586 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.880563    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/746a745b-0cf6-4b92-8173-4a8932ec5dff-registration-dir\") pod \"aws-ebs-csi-driver-node-w779p\" (UID: \"746a745b-0cf6-4b92-8173-4a8932ec5dff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w779p"
Jul 28 19:58:58.880693 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.880590    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/95c98833-0ec7-45eb-8986-14d40c9986e8-etc-sysctl-conf\") pod \"tuned-9lw2g\" (UID: \"95c98833-0ec7-45eb-8986-14d40c9986e8\") " pod="openshift-cluster-node-tuning-operator/tuned-9lw2g"
Jul 28 19:58:58.880693 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.880606    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/752faedf-0642-47aa-88bb-72f0ae961ebb-metrics-certs\") pod \"network-metrics-daemon-xmd9l\" (UID: \"752faedf-0642-47aa-88bb-72f0ae961ebb\") " pod="openshift-multus/network-metrics-daemon-xmd9l"
Jul 28 19:58:58.880693 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.880621    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/38cf5bd2-15db-49bc-b41d-b0df3ea06cb1-host\") pod \"node-ca-qrztd\" (UID: \"38cf5bd2-15db-49bc-b41d-b0df3ea06cb1\") " pod="openshift-image-registry/node-ca-qrztd"
Jul 28 19:58:58.880693 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.880637    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/95c98833-0ec7-45eb-8986-14d40c9986e8-sys\") pod \"tuned-9lw2g\" (UID: \"95c98833-0ec7-45eb-8986-14d40c9986e8\") " pod="openshift-cluster-node-tuning-operator/tuned-9lw2g"
Jul 28 19:58:58.880693 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.880651    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/95c98833-0ec7-45eb-8986-14d40c9986e8-lib-modules\") pod \"tuned-9lw2g\" (UID: \"95c98833-0ec7-45eb-8986-14d40c9986e8\") " pod="openshift-cluster-node-tuning-operator/tuned-9lw2g"
Jul 28 19:58:58.880693 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.880679    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6485753f-f774-4aa4-aaec-130c7b7ba868-system-cni-dir\") pod \"multus-xxgzp\" (UID: \"6485753f-f774-4aa4-aaec-130c7b7ba868\") " pod="openshift-multus/multus-xxgzp"
Jul 28 19:58:58.880992 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.880702    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gx92\" (UniqueName: \"kubernetes.io/projected/752faedf-0642-47aa-88bb-72f0ae961ebb-kube-api-access-5gx92\") pod \"network-metrics-daemon-xmd9l\" (UID: \"752faedf-0642-47aa-88bb-72f0ae961ebb\") " pod="openshift-multus/network-metrics-daemon-xmd9l"
Jul 28 19:58:58.880992 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.880714    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/38cf5bd2-15db-49bc-b41d-b0df3ea06cb1-host\") pod \"node-ca-qrztd\" (UID: \"38cf5bd2-15db-49bc-b41d-b0df3ea06cb1\") " pod="openshift-image-registry/node-ca-qrztd"
Jul 28 19:58:58.880992 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:58:58.880737    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:58:58.880992 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.880715    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/746a745b-0cf6-4b92-8173-4a8932ec5dff-registration-dir\") pod \"aws-ebs-csi-driver-node-w779p\" (UID: \"746a745b-0cf6-4b92-8173-4a8932ec5dff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w779p"
Jul 28 19:58:58.880992 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.880770    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6485753f-f774-4aa4-aaec-130c7b7ba868-system-cni-dir\") pod \"multus-xxgzp\" (UID: \"6485753f-f774-4aa4-aaec-130c7b7ba868\") " pod="openshift-multus/multus-xxgzp"
Jul 28 19:58:58.880992 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.880784    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/95c98833-0ec7-45eb-8986-14d40c9986e8-etc-systemd\") pod \"tuned-9lw2g\" (UID: \"95c98833-0ec7-45eb-8986-14d40c9986e8\") " pod="openshift-cluster-node-tuning-operator/tuned-9lw2g"
Jul 28 19:58:58.880992 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:58:58.880794    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/752faedf-0642-47aa-88bb-72f0ae961ebb-metrics-certs podName:752faedf-0642-47aa-88bb-72f0ae961ebb nodeName:}" failed. No retries permitted until 2026-07-28 19:58:59.380775678 +0000 UTC m=+2.163787123 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/752faedf-0642-47aa-88bb-72f0ae961ebb-metrics-certs") pod "network-metrics-daemon-xmd9l" (UID: "752faedf-0642-47aa-88bb-72f0ae961ebb") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:58:58.880992 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.880724    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/95c98833-0ec7-45eb-8986-14d40c9986e8-etc-systemd\") pod \"tuned-9lw2g\" (UID: \"95c98833-0ec7-45eb-8986-14d40c9986e8\") " pod="openshift-cluster-node-tuning-operator/tuned-9lw2g"
Jul 28 19:58:58.880992 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.880824    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/95c98833-0ec7-45eb-8986-14d40c9986e8-sys\") pod \"tuned-9lw2g\" (UID: \"95c98833-0ec7-45eb-8986-14d40c9986e8\") " pod="openshift-cluster-node-tuning-operator/tuned-9lw2g"
Jul 28 19:58:58.880992 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.880810    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/95c98833-0ec7-45eb-8986-14d40c9986e8-lib-modules\") pod \"tuned-9lw2g\" (UID: \"95c98833-0ec7-45eb-8986-14d40c9986e8\") " pod="openshift-cluster-node-tuning-operator/tuned-9lw2g"
Jul 28 19:58:58.880992 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.880839    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6485753f-f774-4aa4-aaec-130c7b7ba868-multus-cni-dir\") pod \"multus-xxgzp\" (UID: \"6485753f-f774-4aa4-aaec-130c7b7ba868\") " pod="openshift-multus/multus-xxgzp"
Jul 28 19:58:58.880992 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.880868    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6485753f-f774-4aa4-aaec-130c7b7ba868-host-run-multus-certs\") pod \"multus-xxgzp\" (UID: \"6485753f-f774-4aa4-aaec-130c7b7ba868\") " pod="openshift-multus/multus-xxgzp"
Jul 28 19:58:58.880992 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.880885    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6485753f-f774-4aa4-aaec-130c7b7ba868-multus-cni-dir\") pod \"multus-xxgzp\" (UID: \"6485753f-f774-4aa4-aaec-130c7b7ba868\") " pod="openshift-multus/multus-xxgzp"
Jul 28 19:58:58.880992 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.880894    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/95c98833-0ec7-45eb-8986-14d40c9986e8-etc-sysctl-conf\") pod \"tuned-9lw2g\" (UID: \"95c98833-0ec7-45eb-8986-14d40c9986e8\") " pod="openshift-cluster-node-tuning-operator/tuned-9lw2g"
Jul 28 19:58:58.880992 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.880896    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6485753f-f774-4aa4-aaec-130c7b7ba868-host-var-lib-cni-multus\") pod \"multus-xxgzp\" (UID: \"6485753f-f774-4aa4-aaec-130c7b7ba868\") " pod="openshift-multus/multus-xxgzp"
Jul 28 19:58:58.880992 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.880923    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6485753f-f774-4aa4-aaec-130c7b7ba868-host-run-multus-certs\") pod \"multus-xxgzp\" (UID: \"6485753f-f774-4aa4-aaec-130c7b7ba868\") " pod="openshift-multus/multus-xxgzp"
Jul 28 19:58:58.880992 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.880932    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6485753f-f774-4aa4-aaec-130c7b7ba868-host-var-lib-cni-multus\") pod \"multus-xxgzp\" (UID: \"6485753f-f774-4aa4-aaec-130c7b7ba868\") " pod="openshift-multus/multus-xxgzp"
Jul 28 19:58:58.881824 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.880942    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7lx9\" (UniqueName: \"kubernetes.io/projected/38cf5bd2-15db-49bc-b41d-b0df3ea06cb1-kube-api-access-p7lx9\") pod \"node-ca-qrztd\" (UID: \"38cf5bd2-15db-49bc-b41d-b0df3ea06cb1\") " pod="openshift-image-registry/node-ca-qrztd"
Jul 28 19:58:58.881824 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.880975    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/746a745b-0cf6-4b92-8173-4a8932ec5dff-kubelet-dir\") pod \"aws-ebs-csi-driver-node-w779p\" (UID: \"746a745b-0cf6-4b92-8173-4a8932ec5dff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w779p"
Jul 28 19:58:58.881824 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.880995    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6485753f-f774-4aa4-aaec-130c7b7ba868-hostroot\") pod \"multus-xxgzp\" (UID: \"6485753f-f774-4aa4-aaec-130c7b7ba868\") " pod="openshift-multus/multus-xxgzp"
Jul 28 19:58:58.881824 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.881010    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/03099009-562b-4d4b-9db8-95ec824e4b1c-iptables-alerter-script\") pod \"iptables-alerter-9qlvt\" (UID: \"03099009-562b-4d4b-9db8-95ec824e4b1c\") " pod="openshift-network-operator/iptables-alerter-9qlvt"
Jul 28 19:58:58.881824 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.881037    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/95c98833-0ec7-45eb-8986-14d40c9986e8-etc-sysctl-d\") pod \"tuned-9lw2g\" (UID: \"95c98833-0ec7-45eb-8986-14d40c9986e8\") " pod="openshift-cluster-node-tuning-operator/tuned-9lw2g"
Jul 28 19:58:58.881824 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.881046    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6485753f-f774-4aa4-aaec-130c7b7ba868-hostroot\") pod \"multus-xxgzp\" (UID: \"6485753f-f774-4aa4-aaec-130c7b7ba868\") " pod="openshift-multus/multus-xxgzp"
Jul 28 19:58:58.881824 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.881054    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/746a745b-0cf6-4b92-8173-4a8932ec5dff-kubelet-dir\") pod \"aws-ebs-csi-driver-node-w779p\" (UID: \"746a745b-0cf6-4b92-8173-4a8932ec5dff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w779p"
Jul 28 19:58:58.881824 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.881102    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/95c98833-0ec7-45eb-8986-14d40c9986e8-host\") pod \"tuned-9lw2g\" (UID: \"95c98833-0ec7-45eb-8986-14d40c9986e8\") " pod="openshift-cluster-node-tuning-operator/tuned-9lw2g"
Jul 28 19:58:58.881824 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.881125    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6485753f-f774-4aa4-aaec-130c7b7ba868-cni-binary-copy\") pod \"multus-xxgzp\" (UID: \"6485753f-f774-4aa4-aaec-130c7b7ba868\") " pod="openshift-multus/multus-xxgzp"
Jul 28 19:58:58.881824 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.881138    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/95c98833-0ec7-45eb-8986-14d40c9986e8-etc-sysctl-d\") pod \"tuned-9lw2g\" (UID: \"95c98833-0ec7-45eb-8986-14d40c9986e8\") " pod="openshift-cluster-node-tuning-operator/tuned-9lw2g"
Jul 28 19:58:58.881824 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.881160    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/95c98833-0ec7-45eb-8986-14d40c9986e8-etc-modprobe-d\") pod \"tuned-9lw2g\" (UID: \"95c98833-0ec7-45eb-8986-14d40c9986e8\") " pod="openshift-cluster-node-tuning-operator/tuned-9lw2g"
Jul 28 19:58:58.881824 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.881182    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/95c98833-0ec7-45eb-8986-14d40c9986e8-etc-sysconfig\") pod \"tuned-9lw2g\" (UID: \"95c98833-0ec7-45eb-8986-14d40c9986e8\") " pod="openshift-cluster-node-tuning-operator/tuned-9lw2g"
Jul 28 19:58:58.881824 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.881183    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/95c98833-0ec7-45eb-8986-14d40c9986e8-host\") pod \"tuned-9lw2g\" (UID: \"95c98833-0ec7-45eb-8986-14d40c9986e8\") " pod="openshift-cluster-node-tuning-operator/tuned-9lw2g"
Jul 28 19:58:58.881824 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.881205    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95c98833-0ec7-45eb-8986-14d40c9986e8-etc-kubernetes\") pod \"tuned-9lw2g\" (UID: \"95c98833-0ec7-45eb-8986-14d40c9986e8\") " pod="openshift-cluster-node-tuning-operator/tuned-9lw2g"
Jul 28 19:58:58.881824 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.881220    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/95c98833-0ec7-45eb-8986-14d40c9986e8-etc-sysconfig\") pod \"tuned-9lw2g\" (UID: \"95c98833-0ec7-45eb-8986-14d40c9986e8\") " pod="openshift-cluster-node-tuning-operator/tuned-9lw2g"
Jul 28 19:58:58.881824 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.881247    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/95c98833-0ec7-45eb-8986-14d40c9986e8-etc-modprobe-d\") pod \"tuned-9lw2g\" (UID: \"95c98833-0ec7-45eb-8986-14d40c9986e8\") " pod="openshift-cluster-node-tuning-operator/tuned-9lw2g"
Jul 28 19:58:58.881824 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.881253    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6485753f-f774-4aa4-aaec-130c7b7ba868-host-var-lib-cni-bin\") pod \"multus-xxgzp\" (UID: \"6485753f-f774-4aa4-aaec-130c7b7ba868\") " pod="openshift-multus/multus-xxgzp"
Jul 28 19:58:58.881824 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.881273    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95c98833-0ec7-45eb-8986-14d40c9986e8-etc-kubernetes\") pod \"tuned-9lw2g\" (UID: \"95c98833-0ec7-45eb-8986-14d40c9986e8\") " pod="openshift-cluster-node-tuning-operator/tuned-9lw2g"
Jul 28 19:58:58.882667 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.881290    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6485753f-f774-4aa4-aaec-130c7b7ba868-host-var-lib-cni-bin\") pod \"multus-xxgzp\" (UID: \"6485753f-f774-4aa4-aaec-130c7b7ba868\") " pod="openshift-multus/multus-xxgzp"
Jul 28 19:58:58.882667 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.881295    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6485753f-f774-4aa4-aaec-130c7b7ba868-multus-conf-dir\") pod \"multus-xxgzp\" (UID: \"6485753f-f774-4aa4-aaec-130c7b7ba868\") " pod="openshift-multus/multus-xxgzp"
Jul 28 19:58:58.882667 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.881324    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dt7v\" (UniqueName: \"kubernetes.io/projected/03099009-562b-4d4b-9db8-95ec824e4b1c-kube-api-access-2dt7v\") pod \"iptables-alerter-9qlvt\" (UID: \"03099009-562b-4d4b-9db8-95ec824e4b1c\") " pod="openshift-network-operator/iptables-alerter-9qlvt"
Jul 28 19:58:58.882667 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.881348    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/95c98833-0ec7-45eb-8986-14d40c9986e8-tmp\") pod \"tuned-9lw2g\" (UID: \"95c98833-0ec7-45eb-8986-14d40c9986e8\") " pod="openshift-cluster-node-tuning-operator/tuned-9lw2g"
Jul 28 19:58:58.882667 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.881390    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6485753f-f774-4aa4-aaec-130c7b7ba868-cnibin\") pod \"multus-xxgzp\" (UID: \"6485753f-f774-4aa4-aaec-130c7b7ba868\") " pod="openshift-multus/multus-xxgzp"
Jul 28 19:58:58.882667 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.881385    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6485753f-f774-4aa4-aaec-130c7b7ba868-multus-conf-dir\") pod \"multus-xxgzp\" (UID: \"6485753f-f774-4aa4-aaec-130c7b7ba868\") " pod="openshift-multus/multus-xxgzp"
Jul 28 19:58:58.882667 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.881428    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6485753f-f774-4aa4-aaec-130c7b7ba868-multus-daemon-config\") pod \"multus-xxgzp\" (UID: \"6485753f-f774-4aa4-aaec-130c7b7ba868\") " pod="openshift-multus/multus-xxgzp"
Jul 28 19:58:58.882667 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.881438    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6485753f-f774-4aa4-aaec-130c7b7ba868-cnibin\") pod \"multus-xxgzp\" (UID: \"6485753f-f774-4aa4-aaec-130c7b7ba868\") " pod="openshift-multus/multus-xxgzp"
Jul 28 19:58:58.882667 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.881465    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/95c98833-0ec7-45eb-8986-14d40c9986e8-var-lib-kubelet\") pod \"tuned-9lw2g\" (UID: \"95c98833-0ec7-45eb-8986-14d40c9986e8\") " pod="openshift-cluster-node-tuning-operator/tuned-9lw2g"
Jul 28 19:58:58.882667 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.881490    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkrsp\" (UniqueName: \"kubernetes.io/projected/95c98833-0ec7-45eb-8986-14d40c9986e8-kube-api-access-kkrsp\") pod \"tuned-9lw2g\" (UID: \"95c98833-0ec7-45eb-8986-14d40c9986e8\") " pod="openshift-cluster-node-tuning-operator/tuned-9lw2g"
Jul 28 19:58:58.882667 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.881522    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2wxk\" (UniqueName: \"kubernetes.io/projected/6485753f-f774-4aa4-aaec-130c7b7ba868-kube-api-access-s2wxk\") pod \"multus-xxgzp\" (UID: \"6485753f-f774-4aa4-aaec-130c7b7ba868\") " pod="openshift-multus/multus-xxgzp"
Jul 28 19:58:58.882667 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.881548    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2nbz\" (UniqueName: \"kubernetes.io/projected/746a745b-0cf6-4b92-8173-4a8932ec5dff-kube-api-access-f2nbz\") pod \"aws-ebs-csi-driver-node-w779p\" (UID: \"746a745b-0cf6-4b92-8173-4a8932ec5dff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w779p"
Jul 28 19:58:58.882667 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.881580    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/38cf5bd2-15db-49bc-b41d-b0df3ea06cb1-serviceca\") pod \"node-ca-qrztd\" (UID: \"38cf5bd2-15db-49bc-b41d-b0df3ea06cb1\") " pod="openshift-image-registry/node-ca-qrztd"
Jul 28 19:58:58.882667 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.881599    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/03099009-562b-4d4b-9db8-95ec824e4b1c-iptables-alerter-script\") pod \"iptables-alerter-9qlvt\" (UID: \"03099009-562b-4d4b-9db8-95ec824e4b1c\") " pod="openshift-network-operator/iptables-alerter-9qlvt"
Jul 28 19:58:58.882667 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.881603    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/746a745b-0cf6-4b92-8173-4a8932ec5dff-etc-selinux\") pod \"aws-ebs-csi-driver-node-w779p\" (UID: \"746a745b-0cf6-4b92-8173-4a8932ec5dff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w779p"
Jul 28 19:58:58.882667 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.881649    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/95c98833-0ec7-45eb-8986-14d40c9986e8-etc-tuned\") pod \"tuned-9lw2g\" (UID: \"95c98833-0ec7-45eb-8986-14d40c9986e8\") " pod="openshift-cluster-node-tuning-operator/tuned-9lw2g"
Jul 28 19:58:58.882667 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.881667    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6485753f-f774-4aa4-aaec-130c7b7ba868-host-var-lib-kubelet\") pod \"multus-xxgzp\" (UID: \"6485753f-f774-4aa4-aaec-130c7b7ba868\") " pod="openshift-multus/multus-xxgzp"
Jul 28 19:58:58.882667 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.881678    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/746a745b-0cf6-4b92-8173-4a8932ec5dff-etc-selinux\") pod \"aws-ebs-csi-driver-node-w779p\" (UID: \"746a745b-0cf6-4b92-8173-4a8932ec5dff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w779p"
Jul 28 19:58:58.883289 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.881691    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/746a745b-0cf6-4b92-8173-4a8932ec5dff-device-dir\") pod \"aws-ebs-csi-driver-node-w779p\" (UID: \"746a745b-0cf6-4b92-8173-4a8932ec5dff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w779p"
Jul 28 19:58:58.883289 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.881681    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6485753f-f774-4aa4-aaec-130c7b7ba868-cni-binary-copy\") pod \"multus-xxgzp\" (UID: \"6485753f-f774-4aa4-aaec-130c7b7ba868\") " pod="openshift-multus/multus-xxgzp"
Jul 28 19:58:58.883289 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.881730    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/95c98833-0ec7-45eb-8986-14d40c9986e8-var-lib-kubelet\") pod \"tuned-9lw2g\" (UID: \"95c98833-0ec7-45eb-8986-14d40c9986e8\") " pod="openshift-cluster-node-tuning-operator/tuned-9lw2g"
Jul 28 19:58:58.883289 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.881748    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/746a745b-0cf6-4b92-8173-4a8932ec5dff-socket-dir\") pod \"aws-ebs-csi-driver-node-w779p\" (UID: \"746a745b-0cf6-4b92-8173-4a8932ec5dff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w779p"
Jul 28 19:58:58.883289 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.881770    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/746a745b-0cf6-4b92-8173-4a8932ec5dff-device-dir\") pod \"aws-ebs-csi-driver-node-w779p\" (UID: \"746a745b-0cf6-4b92-8173-4a8932ec5dff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w779p"
Jul 28 19:58:58.883289 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.881730    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6485753f-f774-4aa4-aaec-130c7b7ba868-host-var-lib-kubelet\") pod \"multus-xxgzp\" (UID: \"6485753f-f774-4aa4-aaec-130c7b7ba868\") " pod="openshift-multus/multus-xxgzp"
Jul 28 19:58:58.883289 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.881802    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/95c98833-0ec7-45eb-8986-14d40c9986e8-run\") pod \"tuned-9lw2g\" (UID: \"95c98833-0ec7-45eb-8986-14d40c9986e8\") " pod="openshift-cluster-node-tuning-operator/tuned-9lw2g"
Jul 28 19:58:58.883289 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.881827    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6485753f-f774-4aa4-aaec-130c7b7ba868-host-run-netns\") pod \"multus-xxgzp\" (UID: \"6485753f-f774-4aa4-aaec-130c7b7ba868\") " pod="openshift-multus/multus-xxgzp"
Jul 28 19:58:58.883289 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.881860    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/746a745b-0cf6-4b92-8173-4a8932ec5dff-sys-fs\") pod \"aws-ebs-csi-driver-node-w779p\" (UID: \"746a745b-0cf6-4b92-8173-4a8932ec5dff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w779p"
Jul 28 19:58:58.883289 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.881884    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6485753f-f774-4aa4-aaec-130c7b7ba868-os-release\") pod \"multus-xxgzp\" (UID: \"6485753f-f774-4aa4-aaec-130c7b7ba868\") " pod="openshift-multus/multus-xxgzp"
Jul 28 19:58:58.883289 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.881909    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6485753f-f774-4aa4-aaec-130c7b7ba868-etc-kubernetes\") pod \"multus-xxgzp\" (UID: \"6485753f-f774-4aa4-aaec-130c7b7ba868\") " pod="openshift-multus/multus-xxgzp"
Jul 28 19:58:58.883289 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.881916    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6485753f-f774-4aa4-aaec-130c7b7ba868-multus-daemon-config\") pod \"multus-xxgzp\" (UID: \"6485753f-f774-4aa4-aaec-130c7b7ba868\") " pod="openshift-multus/multus-xxgzp"
Jul 28 19:58:58.883289 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.881937    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/95c98833-0ec7-45eb-8986-14d40c9986e8-run\") pod \"tuned-9lw2g\" (UID: \"95c98833-0ec7-45eb-8986-14d40c9986e8\") " pod="openshift-cluster-node-tuning-operator/tuned-9lw2g"
Jul 28 19:58:58.883289 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.881941    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6485753f-f774-4aa4-aaec-130c7b7ba868-multus-socket-dir-parent\") pod \"multus-xxgzp\" (UID: \"6485753f-f774-4aa4-aaec-130c7b7ba868\") " pod="openshift-multus/multus-xxgzp"
Jul 28 19:58:58.883289 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.881974    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6485753f-f774-4aa4-aaec-130c7b7ba868-host-run-k8s-cni-cncf-io\") pod \"multus-xxgzp\" (UID: \"6485753f-f774-4aa4-aaec-130c7b7ba868\") " pod="openshift-multus/multus-xxgzp"
Jul 28 19:58:58.883289 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.881989    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/746a745b-0cf6-4b92-8173-4a8932ec5dff-sys-fs\") pod \"aws-ebs-csi-driver-node-w779p\" (UID: \"746a745b-0cf6-4b92-8173-4a8932ec5dff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w779p"
Jul 28 19:58:58.883289 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.882025    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6485753f-f774-4aa4-aaec-130c7b7ba868-host-run-k8s-cni-cncf-io\") pod \"multus-xxgzp\" (UID: \"6485753f-f774-4aa4-aaec-130c7b7ba868\") " pod="openshift-multus/multus-xxgzp"
Jul 28 19:58:58.883289 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.882028    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6485753f-f774-4aa4-aaec-130c7b7ba868-host-run-netns\") pod \"multus-xxgzp\" (UID: \"6485753f-f774-4aa4-aaec-130c7b7ba868\") " pod="openshift-multus/multus-xxgzp"
Jul 28 19:58:58.883840 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.882042    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6485753f-f774-4aa4-aaec-130c7b7ba868-os-release\") pod \"multus-xxgzp\" (UID: \"6485753f-f774-4aa4-aaec-130c7b7ba868\") " pod="openshift-multus/multus-xxgzp"
Jul 28 19:58:58.883840 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.882037    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/746a745b-0cf6-4b92-8173-4a8932ec5dff-socket-dir\") pod \"aws-ebs-csi-driver-node-w779p\" (UID: \"746a745b-0cf6-4b92-8173-4a8932ec5dff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w779p"
Jul 28 19:58:58.883840 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.882093    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6485753f-f774-4aa4-aaec-130c7b7ba868-etc-kubernetes\") pod \"multus-xxgzp\" (UID: \"6485753f-f774-4aa4-aaec-130c7b7ba868\") " pod="openshift-multus/multus-xxgzp"
Jul 28 19:58:58.883840 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.882099    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6485753f-f774-4aa4-aaec-130c7b7ba868-multus-socket-dir-parent\") pod \"multus-xxgzp\" (UID: \"6485753f-f774-4aa4-aaec-130c7b7ba868\") " pod="openshift-multus/multus-xxgzp"
Jul 28 19:58:58.883840 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.882128    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/03099009-562b-4d4b-9db8-95ec824e4b1c-host-slash\") pod \"iptables-alerter-9qlvt\" (UID: \"03099009-562b-4d4b-9db8-95ec824e4b1c\") " pod="openshift-network-operator/iptables-alerter-9qlvt"
Jul 28 19:58:58.883840 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.882164    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/38cf5bd2-15db-49bc-b41d-b0df3ea06cb1-serviceca\") pod \"node-ca-qrztd\" (UID: \"38cf5bd2-15db-49bc-b41d-b0df3ea06cb1\") " pod="openshift-image-registry/node-ca-qrztd"
Jul 28 19:58:58.883840 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.882184    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/03099009-562b-4d4b-9db8-95ec824e4b1c-host-slash\") pod \"iptables-alerter-9qlvt\" (UID: \"03099009-562b-4d4b-9db8-95ec824e4b1c\") " pod="openshift-network-operator/iptables-alerter-9qlvt"
Jul 28 19:58:58.883840 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.883748    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/95c98833-0ec7-45eb-8986-14d40c9986e8-tmp\") pod \"tuned-9lw2g\" (UID: \"95c98833-0ec7-45eb-8986-14d40c9986e8\") " pod="openshift-cluster-node-tuning-operator/tuned-9lw2g"
Jul 28 19:58:58.883840 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.883788    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/95c98833-0ec7-45eb-8986-14d40c9986e8-etc-tuned\") pod \"tuned-9lw2g\" (UID: \"95c98833-0ec7-45eb-8986-14d40c9986e8\") " pod="openshift-cluster-node-tuning-operator/tuned-9lw2g"
Jul 28 19:58:58.889824 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.889802    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7lx9\" (UniqueName: \"kubernetes.io/projected/38cf5bd2-15db-49bc-b41d-b0df3ea06cb1-kube-api-access-p7lx9\") pod \"node-ca-qrztd\" (UID: \"38cf5bd2-15db-49bc-b41d-b0df3ea06cb1\") " pod="openshift-image-registry/node-ca-qrztd"
Jul 28 19:58:58.889984 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.889968    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2wxk\" (UniqueName: \"kubernetes.io/projected/6485753f-f774-4aa4-aaec-130c7b7ba868-kube-api-access-s2wxk\") pod \"multus-xxgzp\" (UID: \"6485753f-f774-4aa4-aaec-130c7b7ba868\") " pod="openshift-multus/multus-xxgzp"
Jul 28 19:58:58.890301 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.890279    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkrsp\" (UniqueName: \"kubernetes.io/projected/95c98833-0ec7-45eb-8986-14d40c9986e8-kube-api-access-kkrsp\") pod \"tuned-9lw2g\" (UID: \"95c98833-0ec7-45eb-8986-14d40c9986e8\") " pod="openshift-cluster-node-tuning-operator/tuned-9lw2g"
Jul 28 19:58:58.890376 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.890285    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gx92\" (UniqueName: \"kubernetes.io/projected/752faedf-0642-47aa-88bb-72f0ae961ebb-kube-api-access-5gx92\") pod \"network-metrics-daemon-xmd9l\" (UID: \"752faedf-0642-47aa-88bb-72f0ae961ebb\") " pod="openshift-multus/network-metrics-daemon-xmd9l"
Jul 28 19:58:58.890854 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.890837    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2nbz\" (UniqueName: \"kubernetes.io/projected/746a745b-0cf6-4b92-8173-4a8932ec5dff-kube-api-access-f2nbz\") pod \"aws-ebs-csi-driver-node-w779p\" (UID: \"746a745b-0cf6-4b92-8173-4a8932ec5dff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w779p"
Jul 28 19:58:58.890916 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.890853    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dt7v\" (UniqueName: \"kubernetes.io/projected/03099009-562b-4d4b-9db8-95ec824e4b1c-kube-api-access-2dt7v\") pod \"iptables-alerter-9qlvt\" (UID: \"03099009-562b-4d4b-9db8-95ec824e4b1c\") " pod="openshift-network-operator/iptables-alerter-9qlvt"
Jul 28 19:58:58.990643 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:58.990563    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xs5fh"
Jul 28 19:58:58.998498 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:58.998474    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacc81073_b397_4ed9_8e21_5826814e303e.slice/crio-00f8eea5e5500921d5049d3660aecfa5a1b87f7224bd12259cef638540ac00e0 WatchSource:0}: Error finding container 00f8eea5e5500921d5049d3660aecfa5a1b87f7224bd12259cef638540ac00e0: Status 404 returned error can't find the container with id 00f8eea5e5500921d5049d3660aecfa5a1b87f7224bd12259cef638540ac00e0
Jul 28 19:58:59.013350 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:59.013327    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-42pjq"
Jul 28 19:58:59.020186 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:59.020157    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc254f770_a33c_4a75_bd75_9bdc3e4e12ed.slice/crio-84db99a85134d9e9a4cf53b08da4a4e323935751b3b0402bb6a507e6f499b648 WatchSource:0}: Error finding container 84db99a85134d9e9a4cf53b08da4a4e323935751b3b0402bb6a507e6f499b648: Status 404 returned error can't find the container with id 84db99a85134d9e9a4cf53b08da4a4e323935751b3b0402bb6a507e6f499b648
Jul 28 19:58:59.031152 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:59.031135    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7p48k"
Jul 28 19:58:59.037824 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:59.037803    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea4abf49_55cf_4351_a952_7b7a83e803d9.slice/crio-411bb3d079967a55d2ae5377e535d9d4cb550d8d3e2a776d31435ed01e315952 WatchSource:0}: Error finding container 411bb3d079967a55d2ae5377e535d9d4cb550d8d3e2a776d31435ed01e315952: Status 404 returned error can't find the container with id 411bb3d079967a55d2ae5377e535d9d4cb550d8d3e2a776d31435ed01e315952
Jul 28 19:58:59.050242 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:59.050225    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:58:59.057025 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:59.056994    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8bb4b77_7948_42bb_9433_d048c7205a61.slice/crio-7f01f55e38d80cb212505a75decd5db404ed1247628bb72a63b7ac81ecd0683c WatchSource:0}: Error finding container 7f01f55e38d80cb212505a75decd5db404ed1247628bb72a63b7ac81ecd0683c: Status 404 returned error can't find the container with id 7f01f55e38d80cb212505a75decd5db404ed1247628bb72a63b7ac81ecd0683c
Jul 28 19:58:59.060952 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:59.060935    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qrztd"
Jul 28 19:58:59.067851 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:59.067831    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38cf5bd2_15db_49bc_b41d_b0df3ea06cb1.slice/crio-87720f4b1878f3d2252f5d4a063ead54414ee4f47acbb7b7292290c310838391 WatchSource:0}: Error finding container 87720f4b1878f3d2252f5d4a063ead54414ee4f47acbb7b7292290c310838391: Status 404 returned error can't find the container with id 87720f4b1878f3d2252f5d4a063ead54414ee4f47acbb7b7292290c310838391
Jul 28 19:58:59.083431 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:59.083405    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9qlvt"
Jul 28 19:58:59.089706 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:59.089687    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03099009_562b_4d4b_9db8_95ec824e4b1c.slice/crio-fde4bc86090a40d2b789401ed271ee3ae4d4b1a2ddec95e7ab53ec51ca3fc6ff WatchSource:0}: Error finding container fde4bc86090a40d2b789401ed271ee3ae4d4b1a2ddec95e7ab53ec51ca3fc6ff: Status 404 returned error can't find the container with id fde4bc86090a40d2b789401ed271ee3ae4d4b1a2ddec95e7ab53ec51ca3fc6ff
Jul 28 19:58:59.097945 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:59.097922    2554 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 19:58:59.105188 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:59.105170    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w779p"
Jul 28 19:58:59.112139 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:59.112118    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod746a745b_0cf6_4b92_8173_4a8932ec5dff.slice/crio-4a104f71385953f087d4382ff168f9b15aa44f3f839837e7e3658d33bcda27fc WatchSource:0}: Error finding container 4a104f71385953f087d4382ff168f9b15aa44f3f839837e7e3658d33bcda27fc: Status 404 returned error can't find the container with id 4a104f71385953f087d4382ff168f9b15aa44f3f839837e7e3658d33bcda27fc
Jul 28 19:58:59.122010 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:59.121992    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9lw2g"
Jul 28 19:58:59.126593 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:59.126577    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-xxgzp"
Jul 28 19:58:59.129510 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:59.129485    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95c98833_0ec7_45eb_8986_14d40c9986e8.slice/crio-2fdc145f42eaed11ef48738ed3c6efd9930af8b8f7e69c44f2647e976e35322c WatchSource:0}: Error finding container 2fdc145f42eaed11ef48738ed3c6efd9930af8b8f7e69c44f2647e976e35322c: Status 404 returned error can't find the container with id 2fdc145f42eaed11ef48738ed3c6efd9930af8b8f7e69c44f2647e976e35322c
Jul 28 19:58:59.133854 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:58:59.133830    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6485753f_f774_4aa4_aaec_130c7b7ba868.slice/crio-d1a3832029e14760286ba90386cff02688f9855aebce396fdd68ac3c026d1ded WatchSource:0}: Error finding container d1a3832029e14760286ba90386cff02688f9855aebce396fdd68ac3c026d1ded: Status 404 returned error can't find the container with id d1a3832029e14760286ba90386cff02688f9855aebce396fdd68ac3c026d1ded
Jul 28 19:58:59.285961 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:59.285880    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcjrs\" (UniqueName: \"kubernetes.io/projected/2f157e54-c70e-46b1-a41b-d60279cd7189-kube-api-access-qcjrs\") pod \"network-check-target-v2g2z\" (UID: \"2f157e54-c70e-46b1-a41b-d60279cd7189\") " pod="openshift-network-diagnostics/network-check-target-v2g2z"
Jul 28 19:58:59.286132 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:59.285962    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9cc5727b-a071-4793-a0b1-b92f6287080a-original-pull-secret\") pod \"global-pull-secret-syncer-rv7fr\" (UID: \"9cc5727b-a071-4793-a0b1-b92f6287080a\") " pod="kube-system/global-pull-secret-syncer-rv7fr"
Jul 28 19:58:59.286132 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:58:59.286094    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:58:59.286132 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:58:59.286102    2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:58:59.286132 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:58:59.286119    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:58:59.286132 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:58:59.286132    2554 projected.go:196] Error preparing data for projected volume kube-api-access-qcjrs for pod openshift-network-diagnostics/network-check-target-v2g2z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:58:59.286361 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:58:59.286186    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9cc5727b-a071-4793-a0b1-b92f6287080a-original-pull-secret podName:9cc5727b-a071-4793-a0b1-b92f6287080a nodeName:}" failed. No retries permitted until 2026-07-28 19:59:00.286167403 +0000 UTC m=+3.069178836 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9cc5727b-a071-4793-a0b1-b92f6287080a-original-pull-secret") pod "global-pull-secret-syncer-rv7fr" (UID: "9cc5727b-a071-4793-a0b1-b92f6287080a") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:58:59.286361 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:58:59.286204    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f157e54-c70e-46b1-a41b-d60279cd7189-kube-api-access-qcjrs podName:2f157e54-c70e-46b1-a41b-d60279cd7189 nodeName:}" failed. No retries permitted until 2026-07-28 19:59:00.286194721 +0000 UTC m=+3.069206147 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-qcjrs" (UniqueName: "kubernetes.io/projected/2f157e54-c70e-46b1-a41b-d60279cd7189-kube-api-access-qcjrs") pod "network-check-target-v2g2z" (UID: "2f157e54-c70e-46b1-a41b-d60279cd7189") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:58:59.387243 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:59.387207    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/752faedf-0642-47aa-88bb-72f0ae961ebb-metrics-certs\") pod \"network-metrics-daemon-xmd9l\" (UID: \"752faedf-0642-47aa-88bb-72f0ae961ebb\") " pod="openshift-multus/network-metrics-daemon-xmd9l"
Jul 28 19:58:59.387449 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:58:59.387423    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:58:59.387644 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:58:59.387524    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/752faedf-0642-47aa-88bb-72f0ae961ebb-metrics-certs podName:752faedf-0642-47aa-88bb-72f0ae961ebb nodeName:}" failed. No retries permitted until 2026-07-28 19:59:00.387503031 +0000 UTC m=+3.170514465 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/752faedf-0642-47aa-88bb-72f0ae961ebb-metrics-certs") pod "network-metrics-daemon-xmd9l" (UID: "752faedf-0642-47aa-88bb-72f0ae961ebb") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:58:59.441450 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:59.441239    2554 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 19:58:59.715115 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:59.714973    2554 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 19:53:58 +0000 UTC" deadline="2028-03-28 14:48:08.340946559 +0000 UTC"
Jul 28 19:58:59.715115 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:59.715006    2554 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14610h49m8.625945095s"
Jul 28 19:58:59.783495 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:59.782620    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v2g2z"
Jul 28 19:58:59.783495 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:58:59.782790    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v2g2z" podUID="2f157e54-c70e-46b1-a41b-d60279cd7189"
Jul 28 19:58:59.783495 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:59.783296    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xmd9l"
Jul 28 19:58:59.783495 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:58:59.783440    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xmd9l" podUID="752faedf-0642-47aa-88bb-72f0ae961ebb"
Jul 28 19:58:59.804744 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:59.804269    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9lw2g" event={"ID":"95c98833-0ec7-45eb-8986-14d40c9986e8","Type":"ContainerStarted","Data":"2fdc145f42eaed11ef48738ed3c6efd9930af8b8f7e69c44f2647e976e35322c"}
Jul 28 19:58:59.812458 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:59.812424    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w779p" event={"ID":"746a745b-0cf6-4b92-8173-4a8932ec5dff","Type":"ContainerStarted","Data":"4a104f71385953f087d4382ff168f9b15aa44f3f839837e7e3658d33bcda27fc"}
Jul 28 19:58:59.821710 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:59.821676    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9qlvt" event={"ID":"03099009-562b-4d4b-9db8-95ec824e4b1c","Type":"ContainerStarted","Data":"fde4bc86090a40d2b789401ed271ee3ae4d4b1a2ddec95e7ab53ec51ca3fc6ff"}
Jul 28 19:58:59.860765 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:59.860717    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qxw76" event={"ID":"f8bb4b77-7948-42bb-9433-d048c7205a61","Type":"ContainerStarted","Data":"7f01f55e38d80cb212505a75decd5db404ed1247628bb72a63b7ac81ecd0683c"}
Jul 28 19:58:59.873320 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:59.873286    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-42pjq" event={"ID":"c254f770-a33c-4a75-bd75-9bdc3e4e12ed","Type":"ContainerStarted","Data":"84db99a85134d9e9a4cf53b08da4a4e323935751b3b0402bb6a507e6f499b648"}
Jul 28 19:58:59.883973 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:59.883937    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-54.ec2.internal" event={"ID":"ff4439bcade5003e5697cd6a35723c4e","Type":"ContainerStarted","Data":"bbba22234fa1c9b6d825a3bee300390e7623a18d1e5b808106523b0125985d6e"}
Jul 28 19:58:59.902018 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:59.901983    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-xxgzp" event={"ID":"6485753f-f774-4aa4-aaec-130c7b7ba868","Type":"ContainerStarted","Data":"d1a3832029e14760286ba90386cff02688f9855aebce396fdd68ac3c026d1ded"}
Jul 28 19:58:59.921441 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:59.921406    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qrztd" event={"ID":"38cf5bd2-15db-49bc-b41d-b0df3ea06cb1","Type":"ContainerStarted","Data":"87720f4b1878f3d2252f5d4a063ead54414ee4f47acbb7b7292290c310838391"}
Jul 28 19:58:59.951004 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:59.950966    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7p48k" event={"ID":"ea4abf49-55cf-4351-a952-7b7a83e803d9","Type":"ContainerStarted","Data":"411bb3d079967a55d2ae5377e535d9d4cb550d8d3e2a776d31435ed01e315952"}
Jul 28 19:58:59.965953 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:59.965874    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xs5fh" event={"ID":"acc81073-b397-4ed9-8e21-5826814e303e","Type":"ContainerStarted","Data":"00f8eea5e5500921d5049d3660aecfa5a1b87f7224bd12259cef638540ac00e0"}
Jul 28 19:58:59.980697 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:58:59.980612    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-54.ec2.internal" event={"ID":"74845271ca622f52b49ec9ab9ec8f7b9","Type":"ContainerStarted","Data":"86ffd49c435be2460d107807187acb6ad2a0cac9e79f852b30379d3bcb00041c"}
Jul 28 19:59:00.293724 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:00.293628    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9cc5727b-a071-4793-a0b1-b92f6287080a-original-pull-secret\") pod \"global-pull-secret-syncer-rv7fr\" (UID: \"9cc5727b-a071-4793-a0b1-b92f6287080a\") " pod="kube-system/global-pull-secret-syncer-rv7fr"
Jul 28 19:59:00.293724 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:00.293714    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcjrs\" (UniqueName: \"kubernetes.io/projected/2f157e54-c70e-46b1-a41b-d60279cd7189-kube-api-access-qcjrs\") pod \"network-check-target-v2g2z\" (UID: \"2f157e54-c70e-46b1-a41b-d60279cd7189\") " pod="openshift-network-diagnostics/network-check-target-v2g2z"
Jul 28 19:59:00.293958 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:00.293861    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:59:00.293958 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:00.293882    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:59:00.293958 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:00.293895    2554 projected.go:196] Error preparing data for projected volume kube-api-access-qcjrs for pod openshift-network-diagnostics/network-check-target-v2g2z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:59:00.293958 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:00.293954    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f157e54-c70e-46b1-a41b-d60279cd7189-kube-api-access-qcjrs podName:2f157e54-c70e-46b1-a41b-d60279cd7189 nodeName:}" failed. No retries permitted until 2026-07-28 19:59:02.293933819 +0000 UTC m=+5.076945245 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-qcjrs" (UniqueName: "kubernetes.io/projected/2f157e54-c70e-46b1-a41b-d60279cd7189-kube-api-access-qcjrs") pod "network-check-target-v2g2z" (UID: "2f157e54-c70e-46b1-a41b-d60279cd7189") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:59:00.294385 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:00.294366    2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:59:00.294498 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:00.294421    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9cc5727b-a071-4793-a0b1-b92f6287080a-original-pull-secret podName:9cc5727b-a071-4793-a0b1-b92f6287080a nodeName:}" failed. No retries permitted until 2026-07-28 19:59:02.294405913 +0000 UTC m=+5.077417349 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9cc5727b-a071-4793-a0b1-b92f6287080a-original-pull-secret") pod "global-pull-secret-syncer-rv7fr" (UID: "9cc5727b-a071-4793-a0b1-b92f6287080a") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:59:00.394519 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:00.394479    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/752faedf-0642-47aa-88bb-72f0ae961ebb-metrics-certs\") pod \"network-metrics-daemon-xmd9l\" (UID: \"752faedf-0642-47aa-88bb-72f0ae961ebb\") " pod="openshift-multus/network-metrics-daemon-xmd9l"
Jul 28 19:59:00.394707 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:00.394634    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:59:00.394778 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:00.394715    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/752faedf-0642-47aa-88bb-72f0ae961ebb-metrics-certs podName:752faedf-0642-47aa-88bb-72f0ae961ebb nodeName:}" failed. No retries permitted until 2026-07-28 19:59:02.394694281 +0000 UTC m=+5.177705703 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/752faedf-0642-47aa-88bb-72f0ae961ebb-metrics-certs") pod "network-metrics-daemon-xmd9l" (UID: "752faedf-0642-47aa-88bb-72f0ae961ebb") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:59:00.715379 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:00.715260    2554 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 19:53:58 +0000 UTC" deadline="2028-04-10 16:54:10.245166414 +0000 UTC"
Jul 28 19:59:00.715379 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:00.715319    2554 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14924h55m9.529851034s"
Jul 28 19:59:00.771557 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:00.771519    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rv7fr"
Jul 28 19:59:00.771762 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:00.771664    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rv7fr" podUID="9cc5727b-a071-4793-a0b1-b92f6287080a"
Jul 28 19:59:01.559821 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:01.559789    2554 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 19:59:01.773113 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:01.771558    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v2g2z"
Jul 28 19:59:01.773113 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:01.771694    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v2g2z" podUID="2f157e54-c70e-46b1-a41b-d60279cd7189"
Jul 28 19:59:01.773113 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:01.772127    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xmd9l"
Jul 28 19:59:01.773113 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:01.772246    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xmd9l" podUID="752faedf-0642-47aa-88bb-72f0ae961ebb"
Jul 28 19:59:02.316325 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:02.315488    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcjrs\" (UniqueName: \"kubernetes.io/projected/2f157e54-c70e-46b1-a41b-d60279cd7189-kube-api-access-qcjrs\") pod \"network-check-target-v2g2z\" (UID: \"2f157e54-c70e-46b1-a41b-d60279cd7189\") " pod="openshift-network-diagnostics/network-check-target-v2g2z"
Jul 28 19:59:02.316325 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:02.315563    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9cc5727b-a071-4793-a0b1-b92f6287080a-original-pull-secret\") pod \"global-pull-secret-syncer-rv7fr\" (UID: \"9cc5727b-a071-4793-a0b1-b92f6287080a\") " pod="kube-system/global-pull-secret-syncer-rv7fr"
Jul 28 19:59:02.316325 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:02.315712    2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:59:02.316325 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:02.315775    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9cc5727b-a071-4793-a0b1-b92f6287080a-original-pull-secret podName:9cc5727b-a071-4793-a0b1-b92f6287080a nodeName:}" failed. No retries permitted until 2026-07-28 19:59:06.31575622 +0000 UTC m=+9.098767649 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9cc5727b-a071-4793-a0b1-b92f6287080a-original-pull-secret") pod "global-pull-secret-syncer-rv7fr" (UID: "9cc5727b-a071-4793-a0b1-b92f6287080a") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:59:02.316325 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:02.316209    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:59:02.316325 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:02.316228    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:59:02.316325 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:02.316241    2554 projected.go:196] Error preparing data for projected volume kube-api-access-qcjrs for pod openshift-network-diagnostics/network-check-target-v2g2z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:59:02.316325 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:02.316289    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f157e54-c70e-46b1-a41b-d60279cd7189-kube-api-access-qcjrs podName:2f157e54-c70e-46b1-a41b-d60279cd7189 nodeName:}" failed. No retries permitted until 2026-07-28 19:59:06.316273119 +0000 UTC m=+9.099284543 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-qcjrs" (UniqueName: "kubernetes.io/projected/2f157e54-c70e-46b1-a41b-d60279cd7189-kube-api-access-qcjrs") pod "network-check-target-v2g2z" (UID: "2f157e54-c70e-46b1-a41b-d60279cd7189") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:59:02.416353 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:02.416286    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/752faedf-0642-47aa-88bb-72f0ae961ebb-metrics-certs\") pod \"network-metrics-daemon-xmd9l\" (UID: \"752faedf-0642-47aa-88bb-72f0ae961ebb\") " pod="openshift-multus/network-metrics-daemon-xmd9l"
Jul 28 19:59:02.416508 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:02.416496    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:59:02.416563 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:02.416557    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/752faedf-0642-47aa-88bb-72f0ae961ebb-metrics-certs podName:752faedf-0642-47aa-88bb-72f0ae961ebb nodeName:}" failed. No retries permitted until 2026-07-28 19:59:06.416538678 +0000 UTC m=+9.199550119 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/752faedf-0642-47aa-88bb-72f0ae961ebb-metrics-certs") pod "network-metrics-daemon-xmd9l" (UID: "752faedf-0642-47aa-88bb-72f0ae961ebb") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:59:02.772336 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:02.772298    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rv7fr"
Jul 28 19:59:02.772543 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:02.772438    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rv7fr" podUID="9cc5727b-a071-4793-a0b1-b92f6287080a"
Jul 28 19:59:03.774097 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:03.772505    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v2g2z"
Jul 28 19:59:03.774097 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:03.772632    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v2g2z" podUID="2f157e54-c70e-46b1-a41b-d60279cd7189"
Jul 28 19:59:03.774097 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:03.773046    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xmd9l"
Jul 28 19:59:03.774097 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:03.773206    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xmd9l" podUID="752faedf-0642-47aa-88bb-72f0ae961ebb"
Jul 28 19:59:04.772521 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:04.771984    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rv7fr"
Jul 28 19:59:04.772521 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:04.772138    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rv7fr" podUID="9cc5727b-a071-4793-a0b1-b92f6287080a"
Jul 28 19:59:05.772118 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:05.772085    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xmd9l"
Jul 28 19:59:05.772597 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:05.772237    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xmd9l" podUID="752faedf-0642-47aa-88bb-72f0ae961ebb"
Jul 28 19:59:05.772597 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:05.772565    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v2g2z"
Jul 28 19:59:05.772702 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:05.772652    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v2g2z" podUID="2f157e54-c70e-46b1-a41b-d60279cd7189"
Jul 28 19:59:06.359233 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:06.359193    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcjrs\" (UniqueName: \"kubernetes.io/projected/2f157e54-c70e-46b1-a41b-d60279cd7189-kube-api-access-qcjrs\") pod \"network-check-target-v2g2z\" (UID: \"2f157e54-c70e-46b1-a41b-d60279cd7189\") " pod="openshift-network-diagnostics/network-check-target-v2g2z"
Jul 28 19:59:06.359396 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:06.359277    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9cc5727b-a071-4793-a0b1-b92f6287080a-original-pull-secret\") pod \"global-pull-secret-syncer-rv7fr\" (UID: \"9cc5727b-a071-4793-a0b1-b92f6287080a\") " pod="kube-system/global-pull-secret-syncer-rv7fr"
Jul 28 19:59:06.359494 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:06.359415    2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:59:06.359494 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:06.359479    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9cc5727b-a071-4793-a0b1-b92f6287080a-original-pull-secret podName:9cc5727b-a071-4793-a0b1-b92f6287080a nodeName:}" failed. No retries permitted until 2026-07-28 19:59:14.359462296 +0000 UTC m=+17.142473721 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9cc5727b-a071-4793-a0b1-b92f6287080a-original-pull-secret") pod "global-pull-secret-syncer-rv7fr" (UID: "9cc5727b-a071-4793-a0b1-b92f6287080a") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:59:06.359494 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:06.359415    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:59:06.359644 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:06.359519    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:59:06.359644 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:06.359537    2554 projected.go:196] Error preparing data for projected volume kube-api-access-qcjrs for pod openshift-network-diagnostics/network-check-target-v2g2z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:59:06.359644 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:06.359593    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f157e54-c70e-46b1-a41b-d60279cd7189-kube-api-access-qcjrs podName:2f157e54-c70e-46b1-a41b-d60279cd7189 nodeName:}" failed. No retries permitted until 2026-07-28 19:59:14.359577052 +0000 UTC m=+17.142588495 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-qcjrs" (UniqueName: "kubernetes.io/projected/2f157e54-c70e-46b1-a41b-d60279cd7189-kube-api-access-qcjrs") pod "network-check-target-v2g2z" (UID: "2f157e54-c70e-46b1-a41b-d60279cd7189") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:59:06.460114 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:06.460052    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/752faedf-0642-47aa-88bb-72f0ae961ebb-metrics-certs\") pod \"network-metrics-daemon-xmd9l\" (UID: \"752faedf-0642-47aa-88bb-72f0ae961ebb\") " pod="openshift-multus/network-metrics-daemon-xmd9l"
Jul 28 19:59:06.460300 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:06.460214    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:59:06.460300 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:06.460283    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/752faedf-0642-47aa-88bb-72f0ae961ebb-metrics-certs podName:752faedf-0642-47aa-88bb-72f0ae961ebb nodeName:}" failed. No retries permitted until 2026-07-28 19:59:14.460263776 +0000 UTC m=+17.243275198 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/752faedf-0642-47aa-88bb-72f0ae961ebb-metrics-certs") pod "network-metrics-daemon-xmd9l" (UID: "752faedf-0642-47aa-88bb-72f0ae961ebb") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:59:06.772562 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:06.772464    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rv7fr"
Jul 28 19:59:06.773023 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:06.772613    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rv7fr" podUID="9cc5727b-a071-4793-a0b1-b92f6287080a"
Jul 28 19:59:07.772786 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:07.772758    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xmd9l"
Jul 28 19:59:07.773230 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:07.772858    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xmd9l" podUID="752faedf-0642-47aa-88bb-72f0ae961ebb"
Jul 28 19:59:07.773230 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:07.772973    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v2g2z"
Jul 28 19:59:07.773230 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:07.773096    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v2g2z" podUID="2f157e54-c70e-46b1-a41b-d60279cd7189"
Jul 28 19:59:08.772183 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:08.772142    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rv7fr"
Jul 28 19:59:08.772357 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:08.772282    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rv7fr" podUID="9cc5727b-a071-4793-a0b1-b92f6287080a"
Jul 28 19:59:09.772121 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:09.772091    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xmd9l"
Jul 28 19:59:09.772500 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:09.772200    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xmd9l" podUID="752faedf-0642-47aa-88bb-72f0ae961ebb"
Jul 28 19:59:09.772500 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:09.772257    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v2g2z"
Jul 28 19:59:09.772500 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:09.772342    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v2g2z" podUID="2f157e54-c70e-46b1-a41b-d60279cd7189"
Jul 28 19:59:10.772188 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:10.772149    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rv7fr"
Jul 28 19:59:10.772614 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:10.772277    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rv7fr" podUID="9cc5727b-a071-4793-a0b1-b92f6287080a"
Jul 28 19:59:11.775295 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:11.775264    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v2g2z"
Jul 28 19:59:11.775742 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:11.775379    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v2g2z" podUID="2f157e54-c70e-46b1-a41b-d60279cd7189"
Jul 28 19:59:11.775742 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:11.775427    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xmd9l"
Jul 28 19:59:11.775742 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:11.775560    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xmd9l" podUID="752faedf-0642-47aa-88bb-72f0ae961ebb"
Jul 28 19:59:12.771587 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:12.771552    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rv7fr"
Jul 28 19:59:12.771763 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:12.771688    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rv7fr" podUID="9cc5727b-a071-4793-a0b1-b92f6287080a"
Jul 28 19:59:13.775182 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:13.775034    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xmd9l"
Jul 28 19:59:13.775182 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:13.775048    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v2g2z"
Jul 28 19:59:13.775695 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:13.775190    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xmd9l" podUID="752faedf-0642-47aa-88bb-72f0ae961ebb"
Jul 28 19:59:13.775695 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:13.775336    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v2g2z" podUID="2f157e54-c70e-46b1-a41b-d60279cd7189"
Jul 28 19:59:14.416863 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:14.416825    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9cc5727b-a071-4793-a0b1-b92f6287080a-original-pull-secret\") pod \"global-pull-secret-syncer-rv7fr\" (UID: \"9cc5727b-a071-4793-a0b1-b92f6287080a\") " pod="kube-system/global-pull-secret-syncer-rv7fr"
Jul 28 19:59:14.417095 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:14.416907    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcjrs\" (UniqueName: \"kubernetes.io/projected/2f157e54-c70e-46b1-a41b-d60279cd7189-kube-api-access-qcjrs\") pod \"network-check-target-v2g2z\" (UID: \"2f157e54-c70e-46b1-a41b-d60279cd7189\") " pod="openshift-network-diagnostics/network-check-target-v2g2z"
Jul 28 19:59:14.417095 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:14.416971    2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:59:14.417095 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:14.417052    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9cc5727b-a071-4793-a0b1-b92f6287080a-original-pull-secret podName:9cc5727b-a071-4793-a0b1-b92f6287080a nodeName:}" failed. No retries permitted until 2026-07-28 19:59:30.417031569 +0000 UTC m=+33.200042996 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9cc5727b-a071-4793-a0b1-b92f6287080a-original-pull-secret") pod "global-pull-secret-syncer-rv7fr" (UID: "9cc5727b-a071-4793-a0b1-b92f6287080a") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:59:14.417095 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:14.417051    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:59:14.417095 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:14.417097    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:59:14.417352 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:14.417112    2554 projected.go:196] Error preparing data for projected volume kube-api-access-qcjrs for pod openshift-network-diagnostics/network-check-target-v2g2z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:59:14.417352 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:14.417149    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f157e54-c70e-46b1-a41b-d60279cd7189-kube-api-access-qcjrs podName:2f157e54-c70e-46b1-a41b-d60279cd7189 nodeName:}" failed. No retries permitted until 2026-07-28 19:59:30.417137994 +0000 UTC m=+33.200149436 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-qcjrs" (UniqueName: "kubernetes.io/projected/2f157e54-c70e-46b1-a41b-d60279cd7189-kube-api-access-qcjrs") pod "network-check-target-v2g2z" (UID: "2f157e54-c70e-46b1-a41b-d60279cd7189") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:59:14.517832 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:14.517798    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/752faedf-0642-47aa-88bb-72f0ae961ebb-metrics-certs\") pod \"network-metrics-daemon-xmd9l\" (UID: \"752faedf-0642-47aa-88bb-72f0ae961ebb\") " pod="openshift-multus/network-metrics-daemon-xmd9l"
Jul 28 19:59:14.518015 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:14.517925    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:59:14.518015 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:14.517988    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/752faedf-0642-47aa-88bb-72f0ae961ebb-metrics-certs podName:752faedf-0642-47aa-88bb-72f0ae961ebb nodeName:}" failed. No retries permitted until 2026-07-28 19:59:30.517968658 +0000 UTC m=+33.300980125 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/752faedf-0642-47aa-88bb-72f0ae961ebb-metrics-certs") pod "network-metrics-daemon-xmd9l" (UID: "752faedf-0642-47aa-88bb-72f0ae961ebb") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:59:14.772556 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:14.772471    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rv7fr"
Jul 28 19:59:14.772711 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:14.772601    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rv7fr" podUID="9cc5727b-a071-4793-a0b1-b92f6287080a"
Jul 28 19:59:15.771931 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:15.771898    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v2g2z"
Jul 28 19:59:15.772331 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:15.771897    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xmd9l"
Jul 28 19:59:15.772331 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:15.772009    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v2g2z" podUID="2f157e54-c70e-46b1-a41b-d60279cd7189"
Jul 28 19:59:15.772331 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:15.772095    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xmd9l" podUID="752faedf-0642-47aa-88bb-72f0ae961ebb"
Jul 28 19:59:16.772412 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:16.772163    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rv7fr"
Jul 28 19:59:16.772810 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:16.772520    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rv7fr" podUID="9cc5727b-a071-4793-a0b1-b92f6287080a"
Jul 28 19:59:17.772647 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:17.772607    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xmd9l"
Jul 28 19:59:17.773121 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:17.772692    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v2g2z"
Jul 28 19:59:17.773121 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:17.772713    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xmd9l" podUID="752faedf-0642-47aa-88bb-72f0ae961ebb"
Jul 28 19:59:17.773121 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:17.772826    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v2g2z" podUID="2f157e54-c70e-46b1-a41b-d60279cd7189"
Jul 28 19:59:18.772042 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:18.772017    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rv7fr"
Jul 28 19:59:18.772159 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:18.772141    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rv7fr" podUID="9cc5727b-a071-4793-a0b1-b92f6287080a"
Jul 28 19:59:19.039455 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:19.038109    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9lw2g" event={"ID":"95c98833-0ec7-45eb-8986-14d40c9986e8","Type":"ContainerStarted","Data":"45a9e0816c105ff0cbb917d8aa4be95f5fedafb37e8ad1b877dbabf4bad34f46"}
Jul 28 19:59:19.040907 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:19.040876    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qxw76" event={"ID":"f8bb4b77-7948-42bb-9433-d048c7205a61","Type":"ContainerStarted","Data":"0dfea4424c67f5a2330d83651ad674fb778ec639aa205680745a0e07408a0872"}
Jul 28 19:59:19.040989 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:19.040947    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qxw76" event={"ID":"f8bb4b77-7948-42bb-9433-d048c7205a61","Type":"ContainerStarted","Data":"95dd350065d17453849c1f890d90ecbec790c8f6fd3793f308b44157c7100b3b"}
Jul 28 19:59:19.043050 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:19.043008    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-xxgzp" event={"ID":"6485753f-f774-4aa4-aaec-130c7b7ba868","Type":"ContainerStarted","Data":"2a12ce05eaa6f8e350f22bea93594b0360f2966f52ddc261a037a89997c8e7e5"}
Jul 28 19:59:19.044353 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:19.044335    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-54.ec2.internal" event={"ID":"74845271ca622f52b49ec9ab9ec8f7b9","Type":"ContainerStarted","Data":"e756cf35b1f4976a70c62345ec5f40ca30db476849da1fe2fd692441b77c2c99"}
Jul 28 19:59:19.054270 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:19.054220    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9lw2g" podStartSLOduration=1.52585272 podStartE2EDuration="21.054204838s" totalImagesPullingTime="19.528352118s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:58:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:59:19.053753463 +0000 UTC m=+21.836764906" watchObservedRunningTime="2026-07-28 19:59:19.054204838 +0000 UTC m=+21.837216281"
Jul 28 19:59:19.072177 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:19.072101    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-xxgzp" podStartSLOduration=1.3768013 podStartE2EDuration="21.072079976s" totalImagesPullingTime="19.695278676s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:58:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:59:19.071431597 +0000 UTC m=+21.854443084" watchObservedRunningTime="2026-07-28 19:59:19.072079976 +0000 UTC m=+21.855091420"
Jul 28 19:59:19.089541 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:19.089476    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-54.ec2.internal" podStartSLOduration=21.089454042 podStartE2EDuration="21.089454042s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:58:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:59:19.088692364 +0000 UTC m=+21.871703832" watchObservedRunningTime="2026-07-28 19:59:19.089454042 +0000 UTC m=+21.872465485"
Jul 28 19:59:19.772342 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:19.772307    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v2g2z"
Jul 28 19:59:19.772342 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:19.772307    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xmd9l"
Jul 28 19:59:19.772583 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:19.772427    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v2g2z" podUID="2f157e54-c70e-46b1-a41b-d60279cd7189"
Jul 28 19:59:19.772583 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:19.772545    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xmd9l" podUID="752faedf-0642-47aa-88bb-72f0ae961ebb"
Jul 28 19:59:20.050900 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:20.050832    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w779p" event={"ID":"746a745b-0cf6-4b92-8173-4a8932ec5dff","Type":"ContainerStarted","Data":"3d68122581068acdf1af87ed86a505305c1054e7125d371aa2ac1ede5f9f7a53"}
Jul 28 19:59:20.053776 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:20.053751    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qxw76" event={"ID":"f8bb4b77-7948-42bb-9433-d048c7205a61","Type":"ContainerStarted","Data":"6cec6e8dd36004ca7b446f4b7cfda7e2c9872c612aba909f40994ae4b38fdfd1"}
Jul 28 19:59:20.053892 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:20.053815    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qxw76" event={"ID":"f8bb4b77-7948-42bb-9433-d048c7205a61","Type":"ContainerStarted","Data":"c9be93a9790d0554df86b51d50a29476db2d27fa422b0841a59f230ff771720a"}
Jul 28 19:59:20.053892 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:20.053858    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qxw76" event={"ID":"f8bb4b77-7948-42bb-9433-d048c7205a61","Type":"ContainerStarted","Data":"a05c8d9f879b9f5e8031e061d7204730194f9e1d7193a2c1d0c8157cb7742d8a"}
Jul 28 19:59:20.053892 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:20.053887    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qxw76" event={"ID":"f8bb4b77-7948-42bb-9433-d048c7205a61","Type":"ContainerStarted","Data":"eff88af2971b2948d96078c18bfb1c189943e962bad981c1c1b995636dc1cea5"}
Jul 28 19:59:20.054982 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:20.054960    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-42pjq" event={"ID":"c254f770-a33c-4a75-bd75-9bdc3e4e12ed","Type":"ContainerStarted","Data":"c8076ce87135a142d50e0db31e366c9771a370f8b01733ae0d6de2887e2b82c0"}
Jul 28 19:59:20.056602 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:20.056454    2554 generic.go:356] "Generic (PLEG): container finished" podID="ff4439bcade5003e5697cd6a35723c4e" containerID="2388b158caea772900b89040a2dde5c2b789e398949a1fb0143b232325841cf9" exitCode=0
Jul 28 19:59:20.056681 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:20.056531    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-54.ec2.internal" event={"ID":"ff4439bcade5003e5697cd6a35723c4e","Type":"ContainerDied","Data":"2388b158caea772900b89040a2dde5c2b789e398949a1fb0143b232325841cf9"}
Jul 28 19:59:20.057974 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:20.057945    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qrztd" event={"ID":"38cf5bd2-15db-49bc-b41d-b0df3ea06cb1","Type":"ContainerStarted","Data":"28ddfc3817ec248b7288cbda35fb4e775fba22a41f189dc07175ba48153e4e04"}
Jul 28 19:59:20.059471 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:20.059445    2554 generic.go:356] "Generic (PLEG): container finished" podID="ea4abf49-55cf-4351-a952-7b7a83e803d9" containerID="18b84ffcebf48bade210d381a235703241c6ef88abce8ffef9df67a0bd57858a" exitCode=0
Jul 28 19:59:20.059542 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:20.059515    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7p48k" event={"ID":"ea4abf49-55cf-4351-a952-7b7a83e803d9","Type":"ContainerDied","Data":"18b84ffcebf48bade210d381a235703241c6ef88abce8ffef9df67a0bd57858a"}
Jul 28 19:59:20.060993 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:20.060960    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xs5fh" event={"ID":"acc81073-b397-4ed9-8e21-5826814e303e","Type":"ContainerStarted","Data":"ca5d698c7fe246b3685da6d07ff8fabae09ab99d16e4ec6d91fd3a8561f1579c"}
Jul 28 19:59:20.071484 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:20.071444    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-42pjq" podStartSLOduration=3.432798812 podStartE2EDuration="23.071430071s" totalImagesPullingTime="19.638631259s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:58:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:59:20.070996823 +0000 UTC m=+22.854008264" watchObservedRunningTime="2026-07-28 19:59:20.071430071 +0000 UTC m=+22.854441513"
Jul 28 19:59:20.099749 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:20.099686    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-xs5fh" podStartSLOduration=3.490117405 podStartE2EDuration="23.099664276s" totalImagesPullingTime="19.609546871s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:58:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:59:20.098671224 +0000 UTC m=+22.881682666" watchObservedRunningTime="2026-07-28 19:59:20.099664276 +0000 UTC m=+22.882675922"
Jul 28 19:59:20.112484 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:20.112437    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qrztd" podStartSLOduration=2.547871195 podStartE2EDuration="22.112420506s" totalImagesPullingTime="19.564549311s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:58:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:59:20.111976251 +0000 UTC m=+22.894987733" watchObservedRunningTime="2026-07-28 19:59:20.112420506 +0000 UTC m=+22.895431948"
Jul 28 19:59:20.719561 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:20.719533    2554 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 28 19:59:20.750691 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:20.750606    2554 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-28T19:59:20.719554728Z","UUID":"24f7b4f4-d30d-48d6-9bac-5a75275dcde7","Handler":null,"Name":"","Endpoint":""}
Jul 28 19:59:20.752265 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:20.752241    2554 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 28 19:59:20.752384 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:20.752284    2554 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 28 19:59:20.771698 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:20.771666    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rv7fr"
Jul 28 19:59:20.771825 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:20.771790    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rv7fr" podUID="9cc5727b-a071-4793-a0b1-b92f6287080a"
Jul 28 19:59:21.064851 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:21.064608    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9qlvt" event={"ID":"03099009-562b-4d4b-9db8-95ec824e4b1c","Type":"ContainerStarted","Data":"3a3fb750cdd8fbe7cb0bf76d0c72eed0791678fc3bbfc4f88cb98f7219b6e702"}
Jul 28 19:59:21.066664 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:21.066633    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-54.ec2.internal" event={"ID":"ff4439bcade5003e5697cd6a35723c4e","Type":"ContainerStarted","Data":"b87ecefc3da5575922b19bfb214d1746876748dae61793174102e92a9919ef86"}
Jul 28 19:59:21.068610 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:21.068585    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w779p" event={"ID":"746a745b-0cf6-4b92-8173-4a8932ec5dff","Type":"ContainerStarted","Data":"f19a6057fcfacaa9952bb311c5a7560a62f70d2359e8e0b6b81f2bcffb1ace06"}
Jul 28 19:59:21.079222 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:21.079164    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-9qlvt" podStartSLOduration=3.510933788 podStartE2EDuration="23.079146524s" totalImagesPullingTime="19.568212736s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:58:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:59:21.078799985 +0000 UTC m=+23.861811426" watchObservedRunningTime="2026-07-28 19:59:21.079146524 +0000 UTC m=+23.862157966"
Jul 28 19:59:21.092059 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:21.092008    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-54.ec2.internal" podStartSLOduration=23.091991799 podStartE2EDuration="23.091991799s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:58:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:59:21.091835347 +0000 UTC m=+23.874846809" watchObservedRunningTime="2026-07-28 19:59:21.091991799 +0000 UTC m=+23.875003240"
Jul 28 19:59:21.771889 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:21.771856    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v2g2z"
Jul 28 19:59:21.772057 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:21.771987    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v2g2z" podUID="2f157e54-c70e-46b1-a41b-d60279cd7189"
Jul 28 19:59:21.772323 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:21.772287    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xmd9l"
Jul 28 19:59:21.772451 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:21.772424    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xmd9l" podUID="752faedf-0642-47aa-88bb-72f0ae961ebb"
Jul 28 19:59:22.073643 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:22.073605    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w779p" event={"ID":"746a745b-0cf6-4b92-8173-4a8932ec5dff","Type":"ContainerStarted","Data":"8caf10350127c1913d3987762060ecbe9714caa5e1f22b51756f016cc4d1bb8a"}
Jul 28 19:59:22.077529 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:22.077499    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qxw76" event={"ID":"f8bb4b77-7948-42bb-9433-d048c7205a61","Type":"ContainerStarted","Data":"bcbf9b7c1d76fce1aa9326ae8107fab4e030aa5ae2370926b55d101bff4482b0"}
Jul 28 19:59:22.090208 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:22.090155    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w779p" podStartSLOduration=2.048703206 podStartE2EDuration="24.090140099s" totalImagesPullingTime="22.041436893s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:58:58 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:59:22.090088041 +0000 UTC m=+24.873099503" watchObservedRunningTime="2026-07-28 19:59:22.090140099 +0000 UTC m=+24.873151540"
Jul 28 19:59:22.726756 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:22.726708    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-xs5fh"
Jul 28 19:59:22.727415 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:22.727392    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-xs5fh"
Jul 28 19:59:22.772080 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:22.772017    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rv7fr"
Jul 28 19:59:22.772252 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:22.772154    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rv7fr" podUID="9cc5727b-a071-4793-a0b1-b92f6287080a"
Jul 28 19:59:23.080050 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:23.079975    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-xs5fh"
Jul 28 19:59:23.080577 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:23.080527    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-xs5fh"
Jul 28 19:59:23.771732 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:23.771694    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xmd9l"
Jul 28 19:59:23.771951 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:23.771833    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xmd9l" podUID="752faedf-0642-47aa-88bb-72f0ae961ebb"
Jul 28 19:59:23.771951 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:23.771884    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v2g2z"
Jul 28 19:59:23.772047 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:23.771997    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v2g2z" podUID="2f157e54-c70e-46b1-a41b-d60279cd7189"
Jul 28 19:59:24.772325 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:24.772102    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rv7fr"
Jul 28 19:59:24.772958 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:24.772350    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rv7fr" podUID="9cc5727b-a071-4793-a0b1-b92f6287080a"
Jul 28 19:59:25.087747 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:25.087655    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qxw76" event={"ID":"f8bb4b77-7948-42bb-9433-d048c7205a61","Type":"ContainerStarted","Data":"76a84d60e628d07237f4415c601f61d3cb0ea57d14e30fb74f23b6c4c676a39d"}
Jul 28 19:59:25.087927 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:25.087908    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:59:25.089409 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:25.089379    2554 generic.go:356] "Generic (PLEG): container finished" podID="ea4abf49-55cf-4351-a952-7b7a83e803d9" containerID="c47224873aba6440350732f0276871d38a74d94b07b4106e7eeaa38b20c9edb7" exitCode=0
Jul 28 19:59:25.089537 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:25.089412    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7p48k" event={"ID":"ea4abf49-55cf-4351-a952-7b7a83e803d9","Type":"ContainerDied","Data":"c47224873aba6440350732f0276871d38a74d94b07b4106e7eeaa38b20c9edb7"}
Jul 28 19:59:25.104437 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:25.104415    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:59:25.114630 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:25.113759    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-qxw76" podStartSLOduration=7.500232108 podStartE2EDuration="27.113741676s" totalImagesPullingTime="19.613509568s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:58:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:59:25.112060895 +0000 UTC m=+27.895072337" watchObservedRunningTime="2026-07-28 19:59:25.113741676 +0000 UTC m=+27.896753119"
Jul 28 19:59:25.772225 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:25.772192    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v2g2z"
Jul 28 19:59:25.772383 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:25.772321    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v2g2z" podUID="2f157e54-c70e-46b1-a41b-d60279cd7189"
Jul 28 19:59:25.772873 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:25.772844    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xmd9l"
Jul 28 19:59:25.773008 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:25.772980    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xmd9l" podUID="752faedf-0642-47aa-88bb-72f0ae961ebb"
Jul 28 19:59:26.094922 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:26.094647    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7p48k" event={"ID":"ea4abf49-55cf-4351-a952-7b7a83e803d9","Type":"ContainerStarted","Data":"e50b945a05462bcf6f1873cf3cd8ec75e1c23eacf7b4df1031c6ee001656ca66"}
Jul 28 19:59:26.095476 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:26.095456    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:59:26.095594 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:26.095535    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:59:26.111618 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:26.111587    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 19:59:26.243221 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:26.243185    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rv7fr"]
Jul 28 19:59:26.243365 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:26.243307    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rv7fr"
Jul 28 19:59:26.243410 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:26.243385    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rv7fr" podUID="9cc5727b-a071-4793-a0b1-b92f6287080a"
Jul 28 19:59:26.245969 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:26.245942    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xmd9l"]
Jul 28 19:59:26.246125 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:26.246058    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xmd9l"
Jul 28 19:59:26.246259 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:26.246234    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xmd9l" podUID="752faedf-0642-47aa-88bb-72f0ae961ebb"
Jul 28 19:59:26.248645 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:26.248621    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-v2g2z"]
Jul 28 19:59:26.248742 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:26.248723    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v2g2z"
Jul 28 19:59:26.248855 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:26.248819    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v2g2z" podUID="2f157e54-c70e-46b1-a41b-d60279cd7189"
Jul 28 19:59:27.099334 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:27.099300    2554 generic.go:356] "Generic (PLEG): container finished" podID="ea4abf49-55cf-4351-a952-7b7a83e803d9" containerID="e50b945a05462bcf6f1873cf3cd8ec75e1c23eacf7b4df1031c6ee001656ca66" exitCode=0
Jul 28 19:59:27.099708 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:27.099387    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7p48k" event={"ID":"ea4abf49-55cf-4351-a952-7b7a83e803d9","Type":"ContainerDied","Data":"e50b945a05462bcf6f1873cf3cd8ec75e1c23eacf7b4df1031c6ee001656ca66"}
Jul 28 19:59:27.772354 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:27.772329    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xmd9l"
Jul 28 19:59:27.772488 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:27.772443    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v2g2z"
Jul 28 19:59:27.772488 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:27.772450    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xmd9l" podUID="752faedf-0642-47aa-88bb-72f0ae961ebb"
Jul 28 19:59:27.772613 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:27.772513    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v2g2z" podUID="2f157e54-c70e-46b1-a41b-d60279cd7189"
Jul 28 19:59:27.772613 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:27.772534    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rv7fr"
Jul 28 19:59:27.772613 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:27.772597    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rv7fr" podUID="9cc5727b-a071-4793-a0b1-b92f6287080a"
Jul 28 19:59:28.103995 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:28.103960    2554 generic.go:356] "Generic (PLEG): container finished" podID="ea4abf49-55cf-4351-a952-7b7a83e803d9" containerID="a4702ade5e1bf83c8c6697658919ee01dce46af2d1bee8ede38a37f068541514" exitCode=0
Jul 28 19:59:28.104385 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:28.104043    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7p48k" event={"ID":"ea4abf49-55cf-4351-a952-7b7a83e803d9","Type":"ContainerDied","Data":"a4702ade5e1bf83c8c6697658919ee01dce46af2d1bee8ede38a37f068541514"}
Jul 28 19:59:29.772080 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:29.772035    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rv7fr"
Jul 28 19:59:29.772589 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:29.772175    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rv7fr" podUID="9cc5727b-a071-4793-a0b1-b92f6287080a"
Jul 28 19:59:29.772589 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:29.772528    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xmd9l"
Jul 28 19:59:29.772677 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:29.772610    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xmd9l" podUID="752faedf-0642-47aa-88bb-72f0ae961ebb"
Jul 28 19:59:29.772751 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:29.772739    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v2g2z"
Jul 28 19:59:29.772826 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:29.772807    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v2g2z" podUID="2f157e54-c70e-46b1-a41b-d60279cd7189"
Jul 28 19:59:30.445195 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:30.445161    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcjrs\" (UniqueName: \"kubernetes.io/projected/2f157e54-c70e-46b1-a41b-d60279cd7189-kube-api-access-qcjrs\") pod \"network-check-target-v2g2z\" (UID: \"2f157e54-c70e-46b1-a41b-d60279cd7189\") " pod="openshift-network-diagnostics/network-check-target-v2g2z"
Jul 28 19:59:30.445392 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:30.445226    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9cc5727b-a071-4793-a0b1-b92f6287080a-original-pull-secret\") pod \"global-pull-secret-syncer-rv7fr\" (UID: \"9cc5727b-a071-4793-a0b1-b92f6287080a\") " pod="kube-system/global-pull-secret-syncer-rv7fr"
Jul 28 19:59:30.445392 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:30.445346    2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:59:30.445392 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:30.445358    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:59:30.445392 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:30.445385    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:59:30.445558 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:30.445400    2554 projected.go:196] Error preparing data for projected volume kube-api-access-qcjrs for pod openshift-network-diagnostics/network-check-target-v2g2z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:59:30.445558 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:30.445416    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9cc5727b-a071-4793-a0b1-b92f6287080a-original-pull-secret podName:9cc5727b-a071-4793-a0b1-b92f6287080a nodeName:}" failed. No retries permitted until 2026-07-28 20:00:02.445398662 +0000 UTC m=+65.228410097 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9cc5727b-a071-4793-a0b1-b92f6287080a-original-pull-secret") pod "global-pull-secret-syncer-rv7fr" (UID: "9cc5727b-a071-4793-a0b1-b92f6287080a") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:59:30.445558 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:30.445447    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f157e54-c70e-46b1-a41b-d60279cd7189-kube-api-access-qcjrs podName:2f157e54-c70e-46b1-a41b-d60279cd7189 nodeName:}" failed. No retries permitted until 2026-07-28 20:00:02.44543077 +0000 UTC m=+65.228442189 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-qcjrs" (UniqueName: "kubernetes.io/projected/2f157e54-c70e-46b1-a41b-d60279cd7189-kube-api-access-qcjrs") pod "network-check-target-v2g2z" (UID: "2f157e54-c70e-46b1-a41b-d60279cd7189") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:59:30.545873 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:30.545841    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/752faedf-0642-47aa-88bb-72f0ae961ebb-metrics-certs\") pod \"network-metrics-daemon-xmd9l\" (UID: \"752faedf-0642-47aa-88bb-72f0ae961ebb\") " pod="openshift-multus/network-metrics-daemon-xmd9l"
Jul 28 19:59:30.546037 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:30.545976    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:59:30.546037 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:30.546033    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/752faedf-0642-47aa-88bb-72f0ae961ebb-metrics-certs podName:752faedf-0642-47aa-88bb-72f0ae961ebb nodeName:}" failed. No retries permitted until 2026-07-28 20:00:02.546019305 +0000 UTC m=+65.329030725 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/752faedf-0642-47aa-88bb-72f0ae961ebb-metrics-certs") pod "network-metrics-daemon-xmd9l" (UID: "752faedf-0642-47aa-88bb-72f0ae961ebb") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:59:31.528082 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.527994    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-54.ec2.internal" event="NodeReady"
Jul 28 19:59:31.528502 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.528169    2554 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 28 19:59:31.562997 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.562968    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7bcc689c88-gmt7f"]
Jul 28 19:59:31.569480 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.569453    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7bcc689c88-gmt7f"
Jul 28 19:59:31.572668 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.572494    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 28 19:59:31.572668 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.572581    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 28 19:59:31.572862 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.572736    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6f6rv\""
Jul 28 19:59:31.573048 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.573026    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 28 19:59:31.573366 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.573345    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-xg646"]
Jul 28 19:59:31.579916 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.579832    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-qs4pv"]
Jul 28 19:59:31.580380 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.580357    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xg646"
Jul 28 19:59:31.581545 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.581517    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 28 19:59:31.583210 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.583173    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 28 19:59:31.583329 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.583279    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 28 19:59:31.583498 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.583475    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-pwmd4\""
Jul 28 19:59:31.583765 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.583603    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 28 19:59:31.585292 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.585252    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7bcc689c88-gmt7f"]
Jul 28 19:59:31.585378 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.585322    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qs4pv"
Jul 28 19:59:31.585973 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.585926    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xg646"]
Jul 28 19:59:31.588318 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.588260    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 28 19:59:31.588441 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.588360    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ph26b\""
Jul 28 19:59:31.588519 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.588489    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 28 19:59:31.589459 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.589101    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qs4pv"]
Jul 28 19:59:31.709104 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.709051    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xs5fh_acc81073-b397-4ed9-8e21-5826814e303e/konnectivity-agent/0.log" limit=1024
Jul 28 19:59:31.754147 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.754109    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b7b89ba-b64b-454a-b032-34757974c00a-tmp-dir\") pod \"dns-default-qs4pv\" (UID: \"2b7b89ba-b64b-454a-b032-34757974c00a\") " pod="openshift-dns/dns-default-qs4pv"
Jul 28 19:59:31.754308 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.754183    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0dda64d8-2cc8-4164-afc1-303f060864a8-trusted-ca\") pod \"image-registry-7bcc689c88-gmt7f\" (UID: \"0dda64d8-2cc8-4164-afc1-303f060864a8\") " pod="openshift-image-registry/image-registry-7bcc689c88-gmt7f"
Jul 28 19:59:31.754308 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.754250    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e69543ef-79c9-4a8f-bab6-8e1af517c029-cert\") pod \"ingress-canary-xg646\" (UID: \"e69543ef-79c9-4a8f-bab6-8e1af517c029\") " pod="openshift-ingress-canary/ingress-canary-xg646"
Jul 28 19:59:31.754308 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.754302    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0dda64d8-2cc8-4164-afc1-303f060864a8-ca-trust-extracted\") pod \"image-registry-7bcc689c88-gmt7f\" (UID: \"0dda64d8-2cc8-4164-afc1-303f060864a8\") " pod="openshift-image-registry/image-registry-7bcc689c88-gmt7f"
Jul 28 19:59:31.754447 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.754333    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fgb9\" (UniqueName: \"kubernetes.io/projected/2b7b89ba-b64b-454a-b032-34757974c00a-kube-api-access-4fgb9\") pod \"dns-default-qs4pv\" (UID: \"2b7b89ba-b64b-454a-b032-34757974c00a\") " pod="openshift-dns/dns-default-qs4pv"
Jul 28 19:59:31.754447 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.754363    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0dda64d8-2cc8-4164-afc1-303f060864a8-registry-certificates\") pod \"image-registry-7bcc689c88-gmt7f\" (UID: \"0dda64d8-2cc8-4164-afc1-303f060864a8\") " pod="openshift-image-registry/image-registry-7bcc689c88-gmt7f"
Jul 28 19:59:31.754447 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.754387    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2b7b89ba-b64b-454a-b032-34757974c00a-config-volume\") pod \"dns-default-qs4pv\" (UID: \"2b7b89ba-b64b-454a-b032-34757974c00a\") " pod="openshift-dns/dns-default-qs4pv"
Jul 28 19:59:31.754559 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.754450    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0dda64d8-2cc8-4164-afc1-303f060864a8-image-registry-private-configuration\") pod \"image-registry-7bcc689c88-gmt7f\" (UID: \"0dda64d8-2cc8-4164-afc1-303f060864a8\") " pod="openshift-image-registry/image-registry-7bcc689c88-gmt7f"
Jul 28 19:59:31.754559 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.754478    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0dda64d8-2cc8-4164-afc1-303f060864a8-bound-sa-token\") pod \"image-registry-7bcc689c88-gmt7f\" (UID: \"0dda64d8-2cc8-4164-afc1-303f060864a8\") " pod="openshift-image-registry/image-registry-7bcc689c88-gmt7f"
Jul 28 19:59:31.754559 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.754540    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0dda64d8-2cc8-4164-afc1-303f060864a8-installation-pull-secrets\") pod \"image-registry-7bcc689c88-gmt7f\" (UID: \"0dda64d8-2cc8-4164-afc1-303f060864a8\") " pod="openshift-image-registry/image-registry-7bcc689c88-gmt7f"
Jul 28 19:59:31.754683 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.754570    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccwd5\" (UniqueName: \"kubernetes.io/projected/0dda64d8-2cc8-4164-afc1-303f060864a8-kube-api-access-ccwd5\") pod \"image-registry-7bcc689c88-gmt7f\" (UID: \"0dda64d8-2cc8-4164-afc1-303f060864a8\") " pod="openshift-image-registry/image-registry-7bcc689c88-gmt7f"
Jul 28 19:59:31.754683 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.754602    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2b7b89ba-b64b-454a-b032-34757974c00a-metrics-tls\") pod \"dns-default-qs4pv\" (UID: \"2b7b89ba-b64b-454a-b032-34757974c00a\") " pod="openshift-dns/dns-default-qs4pv"
Jul 28 19:59:31.754683 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.754629    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0dda64d8-2cc8-4164-afc1-303f060864a8-registry-tls\") pod \"image-registry-7bcc689c88-gmt7f\" (UID: \"0dda64d8-2cc8-4164-afc1-303f060864a8\") " pod="openshift-image-registry/image-registry-7bcc689c88-gmt7f"
Jul 28 19:59:31.754683 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.754653    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqpss\" (UniqueName: \"kubernetes.io/projected/e69543ef-79c9-4a8f-bab6-8e1af517c029-kube-api-access-tqpss\") pod \"ingress-canary-xg646\" (UID: \"e69543ef-79c9-4a8f-bab6-8e1af517c029\") " pod="openshift-ingress-canary/ingress-canary-xg646"
Jul 28 19:59:31.772446 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.772416    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v2g2z"
Jul 28 19:59:31.772446 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.772434    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xmd9l"
Jul 28 19:59:31.772667 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.772417    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rv7fr"
Jul 28 19:59:31.776826 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.776226    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2kg9b\""
Jul 28 19:59:31.776826 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.776390    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 19:59:31.776826 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.776228    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 19:59:31.776826 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.776634    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 19:59:31.776826 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.776747    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gjwl7\""
Jul 28 19:59:31.776826 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.776810    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 19:59:31.851235 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.851197    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xs5fh_acc81073-b397-4ed9-8e21-5826814e303e/konnectivity-agent/0.log" limit=1024
Jul 28 19:59:31.855081 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.855043    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0dda64d8-2cc8-4164-afc1-303f060864a8-image-registry-private-configuration\") pod \"image-registry-7bcc689c88-gmt7f\" (UID: \"0dda64d8-2cc8-4164-afc1-303f060864a8\") " pod="openshift-image-registry/image-registry-7bcc689c88-gmt7f"
Jul 28 19:59:31.855201 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.855102    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0dda64d8-2cc8-4164-afc1-303f060864a8-bound-sa-token\") pod \"image-registry-7bcc689c88-gmt7f\" (UID: \"0dda64d8-2cc8-4164-afc1-303f060864a8\") " pod="openshift-image-registry/image-registry-7bcc689c88-gmt7f"
Jul 28 19:59:31.855201 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.855144    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0dda64d8-2cc8-4164-afc1-303f060864a8-installation-pull-secrets\") pod \"image-registry-7bcc689c88-gmt7f\" (UID: \"0dda64d8-2cc8-4164-afc1-303f060864a8\") " pod="openshift-image-registry/image-registry-7bcc689c88-gmt7f"
Jul 28 19:59:31.855201 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.855166    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccwd5\" (UniqueName: \"kubernetes.io/projected/0dda64d8-2cc8-4164-afc1-303f060864a8-kube-api-access-ccwd5\") pod \"image-registry-7bcc689c88-gmt7f\" (UID: \"0dda64d8-2cc8-4164-afc1-303f060864a8\") " pod="openshift-image-registry/image-registry-7bcc689c88-gmt7f"
Jul 28 19:59:31.855201 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.855192    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2b7b89ba-b64b-454a-b032-34757974c00a-metrics-tls\") pod \"dns-default-qs4pv\" (UID: \"2b7b89ba-b64b-454a-b032-34757974c00a\") " pod="openshift-dns/dns-default-qs4pv"
Jul 28 19:59:31.855390 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.855216    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0dda64d8-2cc8-4164-afc1-303f060864a8-registry-tls\") pod \"image-registry-7bcc689c88-gmt7f\" (UID: \"0dda64d8-2cc8-4164-afc1-303f060864a8\") " pod="openshift-image-registry/image-registry-7bcc689c88-gmt7f"
Jul 28 19:59:31.855390 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.855240    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqpss\" (UniqueName: \"kubernetes.io/projected/e69543ef-79c9-4a8f-bab6-8e1af517c029-kube-api-access-tqpss\") pod \"ingress-canary-xg646\" (UID: \"e69543ef-79c9-4a8f-bab6-8e1af517c029\") " pod="openshift-ingress-canary/ingress-canary-xg646"
Jul 28 19:59:31.855390 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.855268    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b7b89ba-b64b-454a-b032-34757974c00a-tmp-dir\") pod \"dns-default-qs4pv\" (UID: \"2b7b89ba-b64b-454a-b032-34757974c00a\") " pod="openshift-dns/dns-default-qs4pv"
Jul 28 19:59:31.855390 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.855314    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0dda64d8-2cc8-4164-afc1-303f060864a8-trusted-ca\") pod \"image-registry-7bcc689c88-gmt7f\" (UID: \"0dda64d8-2cc8-4164-afc1-303f060864a8\") " pod="openshift-image-registry/image-registry-7bcc689c88-gmt7f"
Jul 28 19:59:31.855390 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.855340    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e69543ef-79c9-4a8f-bab6-8e1af517c029-cert\") pod \"ingress-canary-xg646\" (UID: \"e69543ef-79c9-4a8f-bab6-8e1af517c029\") " pod="openshift-ingress-canary/ingress-canary-xg646"
Jul 28 19:59:31.855390 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.855371    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0dda64d8-2cc8-4164-afc1-303f060864a8-ca-trust-extracted\") pod \"image-registry-7bcc689c88-gmt7f\" (UID: \"0dda64d8-2cc8-4164-afc1-303f060864a8\") " pod="openshift-image-registry/image-registry-7bcc689c88-gmt7f"
Jul 28 19:59:31.855657 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.855400    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fgb9\" (UniqueName: \"kubernetes.io/projected/2b7b89ba-b64b-454a-b032-34757974c00a-kube-api-access-4fgb9\") pod \"dns-default-qs4pv\" (UID: \"2b7b89ba-b64b-454a-b032-34757974c00a\") " pod="openshift-dns/dns-default-qs4pv"
Jul 28 19:59:31.855657 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.855431    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0dda64d8-2cc8-4164-afc1-303f060864a8-registry-certificates\") pod \"image-registry-7bcc689c88-gmt7f\" (UID: \"0dda64d8-2cc8-4164-afc1-303f060864a8\") " pod="openshift-image-registry/image-registry-7bcc689c88-gmt7f"
Jul 28 19:59:31.855657 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.855454    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2b7b89ba-b64b-454a-b032-34757974c00a-config-volume\") pod \"dns-default-qs4pv\" (UID: \"2b7b89ba-b64b-454a-b032-34757974c00a\") " pod="openshift-dns/dns-default-qs4pv"
Jul 28 19:59:31.855657 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:31.855513    2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 19:59:31.855657 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:31.855570    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b7b89ba-b64b-454a-b032-34757974c00a-metrics-tls podName:2b7b89ba-b64b-454a-b032-34757974c00a nodeName:}" failed. No retries permitted until 2026-07-28 19:59:32.355551207 +0000 UTC m=+35.138562632 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2b7b89ba-b64b-454a-b032-34757974c00a-metrics-tls") pod "dns-default-qs4pv" (UID: "2b7b89ba-b64b-454a-b032-34757974c00a") : secret "dns-default-metrics-tls" not found
Jul 28 19:59:31.855875 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:31.855759    2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 19:59:31.855875 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:31.855777    2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7bcc689c88-gmt7f: secret "image-registry-tls" not found
Jul 28 19:59:31.855875 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:31.855829    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0dda64d8-2cc8-4164-afc1-303f060864a8-registry-tls podName:0dda64d8-2cc8-4164-afc1-303f060864a8 nodeName:}" failed. No retries permitted until 2026-07-28 19:59:32.355815594 +0000 UTC m=+35.138827014 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0dda64d8-2cc8-4164-afc1-303f060864a8-registry-tls") pod "image-registry-7bcc689c88-gmt7f" (UID: "0dda64d8-2cc8-4164-afc1-303f060864a8") : secret "image-registry-tls" not found
Jul 28 19:59:31.856564 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.856140    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0dda64d8-2cc8-4164-afc1-303f060864a8-ca-trust-extracted\") pod \"image-registry-7bcc689c88-gmt7f\" (UID: \"0dda64d8-2cc8-4164-afc1-303f060864a8\") " pod="openshift-image-registry/image-registry-7bcc689c88-gmt7f"
Jul 28 19:59:31.856564 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.856310    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2b7b89ba-b64b-454a-b032-34757974c00a-config-volume\") pod \"dns-default-qs4pv\" (UID: \"2b7b89ba-b64b-454a-b032-34757974c00a\") " pod="openshift-dns/dns-default-qs4pv"
Jul 28 19:59:31.856564 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.856342    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b7b89ba-b64b-454a-b032-34757974c00a-tmp-dir\") pod \"dns-default-qs4pv\" (UID: \"2b7b89ba-b64b-454a-b032-34757974c00a\") " pod="openshift-dns/dns-default-qs4pv"
Jul 28 19:59:31.856796 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.856670    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0dda64d8-2cc8-4164-afc1-303f060864a8-registry-certificates\") pod \"image-registry-7bcc689c88-gmt7f\" (UID: \"0dda64d8-2cc8-4164-afc1-303f060864a8\") " pod="openshift-image-registry/image-registry-7bcc689c88-gmt7f"
Jul 28 19:59:31.857007 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.856980    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0dda64d8-2cc8-4164-afc1-303f060864a8-trusted-ca\") pod \"image-registry-7bcc689c88-gmt7f\" (UID: \"0dda64d8-2cc8-4164-afc1-303f060864a8\") " pod="openshift-image-registry/image-registry-7bcc689c88-gmt7f"
Jul 28 19:59:31.860014 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.859869    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0dda64d8-2cc8-4164-afc1-303f060864a8-installation-pull-secrets\") pod \"image-registry-7bcc689c88-gmt7f\" (UID: \"0dda64d8-2cc8-4164-afc1-303f060864a8\") " pod="openshift-image-registry/image-registry-7bcc689c88-gmt7f"
Jul 28 19:59:31.860143 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.859869    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0dda64d8-2cc8-4164-afc1-303f060864a8-image-registry-private-configuration\") pod \"image-registry-7bcc689c88-gmt7f\" (UID: \"0dda64d8-2cc8-4164-afc1-303f060864a8\") " pod="openshift-image-registry/image-registry-7bcc689c88-gmt7f"
Jul 28 19:59:31.860143 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.859893    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e69543ef-79c9-4a8f-bab6-8e1af517c029-cert\") pod \"ingress-canary-xg646\" (UID: \"e69543ef-79c9-4a8f-bab6-8e1af517c029\") " pod="openshift-ingress-canary/ingress-canary-xg646"
Jul 28 19:59:31.864653 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.864623    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqpss\" (UniqueName: \"kubernetes.io/projected/e69543ef-79c9-4a8f-bab6-8e1af517c029-kube-api-access-tqpss\") pod \"ingress-canary-xg646\" (UID: \"e69543ef-79c9-4a8f-bab6-8e1af517c029\") " pod="openshift-ingress-canary/ingress-canary-xg646"
Jul 28 19:59:31.865046 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.864987    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fgb9\" (UniqueName: \"kubernetes.io/projected/2b7b89ba-b64b-454a-b032-34757974c00a-kube-api-access-4fgb9\") pod \"dns-default-qs4pv\" (UID: \"2b7b89ba-b64b-454a-b032-34757974c00a\") " pod="openshift-dns/dns-default-qs4pv"
Jul 28 19:59:31.865161 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.865120    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0dda64d8-2cc8-4164-afc1-303f060864a8-bound-sa-token\") pod \"image-registry-7bcc689c88-gmt7f\" (UID: \"0dda64d8-2cc8-4164-afc1-303f060864a8\") " pod="openshift-image-registry/image-registry-7bcc689c88-gmt7f"
Jul 28 19:59:31.865271 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.865252    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccwd5\" (UniqueName: \"kubernetes.io/projected/0dda64d8-2cc8-4164-afc1-303f060864a8-kube-api-access-ccwd5\") pod \"image-registry-7bcc689c88-gmt7f\" (UID: \"0dda64d8-2cc8-4164-afc1-303f060864a8\") " pod="openshift-image-registry/image-registry-7bcc689c88-gmt7f"
Jul 28 19:59:31.895661 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:31.895634    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xg646"
Jul 28 19:59:32.146743 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:32.146708    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xg646"]
Jul 28 19:59:32.233651 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:59:32.233617    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode69543ef_79c9_4a8f_bab6_8e1af517c029.slice/crio-7d9bd090c64a06e916d61c5cc8750e21828b904147cb98c0b072997ce32d7c67 WatchSource:0}: Error finding container 7d9bd090c64a06e916d61c5cc8750e21828b904147cb98c0b072997ce32d7c67: Status 404 returned error can't find the container with id 7d9bd090c64a06e916d61c5cc8750e21828b904147cb98c0b072997ce32d7c67
Jul 28 19:59:32.358834 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:32.358808    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2b7b89ba-b64b-454a-b032-34757974c00a-metrics-tls\") pod \"dns-default-qs4pv\" (UID: \"2b7b89ba-b64b-454a-b032-34757974c00a\") " pod="openshift-dns/dns-default-qs4pv"
Jul 28 19:59:32.358943 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:32.358848    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0dda64d8-2cc8-4164-afc1-303f060864a8-registry-tls\") pod \"image-registry-7bcc689c88-gmt7f\" (UID: \"0dda64d8-2cc8-4164-afc1-303f060864a8\") " pod="openshift-image-registry/image-registry-7bcc689c88-gmt7f"
Jul 28 19:59:32.359008 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:32.358956    2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 19:59:32.359008 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:32.358972    2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 19:59:32.359008 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:32.358988    2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7bcc689c88-gmt7f: secret "image-registry-tls" not found
Jul 28 19:59:32.359143 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:32.359016    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b7b89ba-b64b-454a-b032-34757974c00a-metrics-tls podName:2b7b89ba-b64b-454a-b032-34757974c00a nodeName:}" failed. No retries permitted until 2026-07-28 19:59:33.358999602 +0000 UTC m=+36.142011023 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2b7b89ba-b64b-454a-b032-34757974c00a-metrics-tls") pod "dns-default-qs4pv" (UID: "2b7b89ba-b64b-454a-b032-34757974c00a") : secret "dns-default-metrics-tls" not found
Jul 28 19:59:32.359143 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:32.359037    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0dda64d8-2cc8-4164-afc1-303f060864a8-registry-tls podName:0dda64d8-2cc8-4164-afc1-303f060864a8 nodeName:}" failed. No retries permitted until 2026-07-28 19:59:33.359021797 +0000 UTC m=+36.142033236 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0dda64d8-2cc8-4164-afc1-303f060864a8-registry-tls") pod "image-registry-7bcc689c88-gmt7f" (UID: "0dda64d8-2cc8-4164-afc1-303f060864a8") : secret "image-registry-tls" not found
Jul 28 19:59:33.118602 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:33.118566    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xg646" event={"ID":"e69543ef-79c9-4a8f-bab6-8e1af517c029","Type":"ContainerStarted","Data":"7d9bd090c64a06e916d61c5cc8750e21828b904147cb98c0b072997ce32d7c67"}
Jul 28 19:59:33.121985 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:33.121956    2554 generic.go:356] "Generic (PLEG): container finished" podID="ea4abf49-55cf-4351-a952-7b7a83e803d9" containerID="de075618fac71525873e685e447b0e7cd0a6e29d0feff9309c6549c362e1ec1f" exitCode=0
Jul 28 19:59:33.122143 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:33.122007    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7p48k" event={"ID":"ea4abf49-55cf-4351-a952-7b7a83e803d9","Type":"ContainerDied","Data":"de075618fac71525873e685e447b0e7cd0a6e29d0feff9309c6549c362e1ec1f"}
Jul 28 19:59:33.366926 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:33.366891    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2b7b89ba-b64b-454a-b032-34757974c00a-metrics-tls\") pod \"dns-default-qs4pv\" (UID: \"2b7b89ba-b64b-454a-b032-34757974c00a\") " pod="openshift-dns/dns-default-qs4pv"
Jul 28 19:59:33.366926 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:33.366941    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0dda64d8-2cc8-4164-afc1-303f060864a8-registry-tls\") pod \"image-registry-7bcc689c88-gmt7f\" (UID: \"0dda64d8-2cc8-4164-afc1-303f060864a8\") " pod="openshift-image-registry/image-registry-7bcc689c88-gmt7f"
Jul 28 19:59:33.367184 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:33.367109    2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 19:59:33.367184 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:33.367124    2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7bcc689c88-gmt7f: secret "image-registry-tls" not found
Jul 28 19:59:33.367184 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:33.367130    2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 19:59:33.367184 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:33.367176    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0dda64d8-2cc8-4164-afc1-303f060864a8-registry-tls podName:0dda64d8-2cc8-4164-afc1-303f060864a8 nodeName:}" failed. No retries permitted until 2026-07-28 19:59:35.367157445 +0000 UTC m=+38.150168868 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0dda64d8-2cc8-4164-afc1-303f060864a8-registry-tls") pod "image-registry-7bcc689c88-gmt7f" (UID: "0dda64d8-2cc8-4164-afc1-303f060864a8") : secret "image-registry-tls" not found
Jul 28 19:59:33.367399 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:33.367190    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b7b89ba-b64b-454a-b032-34757974c00a-metrics-tls podName:2b7b89ba-b64b-454a-b032-34757974c00a nodeName:}" failed. No retries permitted until 2026-07-28 19:59:35.367184206 +0000 UTC m=+38.150195626 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2b7b89ba-b64b-454a-b032-34757974c00a-metrics-tls") pod "dns-default-qs4pv" (UID: "2b7b89ba-b64b-454a-b032-34757974c00a") : secret "dns-default-metrics-tls" not found
Jul 28 19:59:34.128227 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:34.128137    2554 generic.go:356] "Generic (PLEG): container finished" podID="ea4abf49-55cf-4351-a952-7b7a83e803d9" containerID="2206a7fef595c4e3284504110e404ca0504ce66332ae863f8a032b8f277c2a0c" exitCode=0
Jul 28 19:59:34.128227 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:34.128215    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7p48k" event={"ID":"ea4abf49-55cf-4351-a952-7b7a83e803d9","Type":"ContainerDied","Data":"2206a7fef595c4e3284504110e404ca0504ce66332ae863f8a032b8f277c2a0c"}
Jul 28 19:59:35.131654 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:35.131622    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xg646" event={"ID":"e69543ef-79c9-4a8f-bab6-8e1af517c029","Type":"ContainerStarted","Data":"cf7fa52c5fd43f9a49be46cbcea3da6d7df3e0103a36d87b359764f813decfc0"}
Jul 28 19:59:35.135147 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:35.135118    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7p48k" event={"ID":"ea4abf49-55cf-4351-a952-7b7a83e803d9","Type":"ContainerStarted","Data":"1906e68948502daea9bf7d50e488e4c36e6a95ac58e68550061f1aebfe30ea62"}
Jul 28 19:59:35.158277 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:35.158219    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-7p48k" podStartSLOduration=8.788248382 podStartE2EDuration="38.158202555s" totalImagesPullingTime="29.369954173s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:58:57 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:59:35.156430172 +0000 UTC m=+37.939441613" watchObservedRunningTime="2026-07-28 19:59:35.158202555 +0000 UTC m=+37.941213996"
Jul 28 19:59:35.386944 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:35.386849    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2b7b89ba-b64b-454a-b032-34757974c00a-metrics-tls\") pod \"dns-default-qs4pv\" (UID: \"2b7b89ba-b64b-454a-b032-34757974c00a\") " pod="openshift-dns/dns-default-qs4pv"
Jul 28 19:59:35.386944 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:35.386892    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0dda64d8-2cc8-4164-afc1-303f060864a8-registry-tls\") pod \"image-registry-7bcc689c88-gmt7f\" (UID: \"0dda64d8-2cc8-4164-afc1-303f060864a8\") " pod="openshift-image-registry/image-registry-7bcc689c88-gmt7f"
Jul 28 19:59:35.387223 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:35.387017    2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 19:59:35.387223 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:35.387022    2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 19:59:35.387223 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:35.387123    2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7bcc689c88-gmt7f: secret "image-registry-tls" not found
Jul 28 19:59:35.387223 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:35.387140    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b7b89ba-b64b-454a-b032-34757974c00a-metrics-tls podName:2b7b89ba-b64b-454a-b032-34757974c00a nodeName:}" failed. No retries permitted until 2026-07-28 19:59:39.387118526 +0000 UTC m=+42.170129947 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2b7b89ba-b64b-454a-b032-34757974c00a-metrics-tls") pod "dns-default-qs4pv" (UID: "2b7b89ba-b64b-454a-b032-34757974c00a") : secret "dns-default-metrics-tls" not found
Jul 28 19:59:35.387223 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:35.387172    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0dda64d8-2cc8-4164-afc1-303f060864a8-registry-tls podName:0dda64d8-2cc8-4164-afc1-303f060864a8 nodeName:}" failed. No retries permitted until 2026-07-28 19:59:39.38716105 +0000 UTC m=+42.170172470 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0dda64d8-2cc8-4164-afc1-303f060864a8-registry-tls") pod "image-registry-7bcc689c88-gmt7f" (UID: "0dda64d8-2cc8-4164-afc1-303f060864a8") : secret "image-registry-tls" not found
Jul 28 19:59:36.152721 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:36.152618    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-xg646" podStartSLOduration=2.381667575 podStartE2EDuration="5.152601754s" totalImagesPullingTime="2.770934179s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:59:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:59:36.151980879 +0000 UTC m=+38.934992322" watchObservedRunningTime="2026-07-28 19:59:36.152601754 +0000 UTC m=+38.935613243"
Jul 28 19:59:36.883426 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:36.883389    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-bqs2c"]
Jul 28 19:59:36.917256 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:36.917229    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-7vcr8"]
Jul 28 19:59:36.917408 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:36.917345    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-bqs2c"
Jul 28 19:59:36.920236 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:36.920201    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 28 19:59:36.921488 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:36.921468    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 28 19:59:36.921488 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:36.921480    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-xqm9w\""
Jul 28 19:59:36.942322 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:36.942299    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-bqs2c"]
Jul 28 19:59:36.942441 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:36.942331    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-7vcr8"]
Jul 28 19:59:36.942441 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:36.942403    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-7vcr8"
Jul 28 19:59:36.945356 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:36.945335    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 28 19:59:36.945524 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:36.945504    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 28 19:59:36.945610 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:36.945533    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 28 19:59:36.945664 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:36.945615    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-h2k96\""
Jul 28 19:59:36.945789 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:36.945775    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 28 19:59:36.950645 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:36.950628    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 28 19:59:36.981064 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:36.981034    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-72w9l"]
Jul 28 19:59:37.000144 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.000099    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/dde4c1a8-8229-4419-87b6-d8e39a0827c4-config\") pod \"console-operator-584dcb6b54-7vcr8\" (UID: \"dde4c1a8-8229-4419-87b6-d8e39a0827c4\") " pod="openshift-console-operator/console-operator-584dcb6b54-7vcr8"
Jul 28 19:59:37.000144 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.000150    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dde4c1a8-8229-4419-87b6-d8e39a0827c4-trusted-ca\") pod \"console-operator-584dcb6b54-7vcr8\" (UID: \"dde4c1a8-8229-4419-87b6-d8e39a0827c4\") " pod="openshift-console-operator/console-operator-584dcb6b54-7vcr8"
Jul 28 19:59:37.000333 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.000171    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6l9xc\" (UniqueName: \"kubernetes.io/projected/dde4c1a8-8229-4419-87b6-d8e39a0827c4-kube-api-access-6l9xc\") pod \"console-operator-584dcb6b54-7vcr8\" (UID: \"dde4c1a8-8229-4419-87b6-d8e39a0827c4\") " pod="openshift-console-operator/console-operator-584dcb6b54-7vcr8"
Jul 28 19:59:37.000333 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.000269    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlkh9\" (UniqueName: \"kubernetes.io/projected/454f1f68-9403-4c21-ba51-55536c8f55c9-kube-api-access-xlkh9\") pod \"volume-data-source-validator-549c6d54cf-bqs2c\" (UID: \"454f1f68-9403-4c21-ba51-55536c8f55c9\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-bqs2c"
Jul 28 19:59:37.000333 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.000319    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/dde4c1a8-8229-4419-87b6-d8e39a0827c4-serving-cert\") pod \"console-operator-584dcb6b54-7vcr8\" (UID: \"dde4c1a8-8229-4419-87b6-d8e39a0827c4\") " pod="openshift-console-operator/console-operator-584dcb6b54-7vcr8"
Jul 28 19:59:37.002372 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.002356    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-xjfzb"]
Jul 28 19:59:37.002476 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.002462    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-72w9l"
Jul 28 19:59:37.005278 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.005177    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-vkqm7\""
Jul 28 19:59:37.014649 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.014623    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-k9b95"]
Jul 28 19:59:37.014800 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.014745    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-xjfzb"
Jul 28 19:59:37.017470 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.017443    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 28 19:59:37.017470 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.017465    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 28 19:59:37.017997 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.017976    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-pnwl5\""
Jul 28 19:59:37.018227 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.018208    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 28 19:59:37.018385 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.018362    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 28 19:59:37.026560 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.026538    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7c5b95cbd9-xk69c"]
Jul 28 19:59:37.026658 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.026631    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-k9b95"
Jul 28 19:59:37.029187 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.029169    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 28 19:59:37.029580 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.029563    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 28 19:59:37.029664 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.029563    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-5pcsr\""
Jul 28 19:59:37.029786 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.029768    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 28 19:59:37.029873 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.029772    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 28 19:59:37.047047 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.047027    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-mfvfs"]
Jul 28 19:59:37.047171 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.047159    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7c5b95cbd9-xk69c"
Jul 28 19:59:37.050330 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.050315    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 28 19:59:37.050847 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.050827    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 28 19:59:37.050897 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.050832    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-v4hr8\""
Jul 28 19:59:37.051008 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.050991    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 28 19:59:37.051100 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.051066    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 28 19:59:37.051168 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.051098    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 28 19:59:37.051306 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.051292    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 28 19:59:37.060644 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.060624    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-72w9l"]
Jul 28 19:59:37.060742 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.060652    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-k9b95"]
Jul 28 19:59:37.060742 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.060665    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-xjfzb"]
Jul 28 19:59:37.060742 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.060679    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7c5b95cbd9-xk69c"]
Jul 28 19:59:37.060742 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.060694    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-mfvfs"]
Jul 28 19:59:37.060934 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.060753    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-mfvfs"
Jul 28 19:59:37.064289 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.064270    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 28 19:59:37.064388 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.064299    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 28 19:59:37.064448 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.064396    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-fq46j\""
Jul 28 19:59:37.064582 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.064567    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 28 19:59:37.064892 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.064878    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 28 19:59:37.071862 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.071842    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 28 19:59:37.099514 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.099485    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-5bj5s"]
Jul 28 19:59:37.100705 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.100684    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ct4cz\" (UniqueName: \"kubernetes.io/projected/d0d19ca1-a117-47c3-a19b-eb756ce208cf-kube-api-access-ct4cz\") pod \"network-check-source-7d88cb55d8-72w9l\" (UID: \"d0d19ca1-a117-47c3-a19b-eb756ce208cf\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-72w9l"
Jul 28 19:59:37.100809 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.100751    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/26b203ca-0b58-4389-ac85-8fdc909e11d2-tmp\") pod \"insights-operator-5885fdb759-mfvfs\" (UID: \"26b203ca-0b58-4389-ac85-8fdc909e11d2\") " pod="openshift-insights/insights-operator-5885fdb759-mfvfs"
Jul 28 19:59:37.100809 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.100789    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26b203ca-0b58-4389-ac85-8fdc909e11d2-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-mfvfs\" (UID: \"26b203ca-0b58-4389-ac85-8fdc909e11d2\") " pod="openshift-insights/insights-operator-5885fdb759-mfvfs"
Jul 28 19:59:37.100901 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.100813    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdfbx\" (UniqueName: \"kubernetes.io/projected/26b203ca-0b58-4389-ac85-8fdc909e11d2-kube-api-access-sdfbx\") pod \"insights-operator-5885fdb759-mfvfs\" (UID: \"26b203ca-0b58-4389-ac85-8fdc909e11d2\") " pod="openshift-insights/insights-operator-5885fdb759-mfvfs"
Jul 28 19:59:37.100901 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.100880    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8f22d663-ad11-402a-a2a6-2f66e778b4ff-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-k9b95\" (UID: \"8f22d663-ad11-402a-a2a6-2f66e778b4ff\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-k9b95"
Jul 28 19:59:37.100968 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.100914    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlkh9\" (UniqueName: \"kubernetes.io/projected/454f1f68-9403-4c21-ba51-55536c8f55c9-kube-api-access-xlkh9\") pod \"volume-data-source-validator-549c6d54cf-bqs2c\" (UID: \"454f1f68-9403-4c21-ba51-55536c8f55c9\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-bqs2c"
Jul 28 19:59:37.100968 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.100958    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/bb24870f-3c94-4836-9e8b-8d6f757c84de-stats-auth\") pod \"router-default-7c5b95cbd9-xk69c\" (UID: \"bb24870f-3c94-4836-9e8b-8d6f757c84de\") " pod="openshift-ingress/router-default-7c5b95cbd9-xk69c"
Jul 28 19:59:37.101027 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.100983    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2d7a85b0-e81f-4686-8889-5096efe82a1a-config\") pod \"service-ca-operator-5d6c75f84f-xjfzb\" (UID: \"2d7a85b0-e81f-4686-8889-5096efe82a1a\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-xjfzb"
Jul 28 19:59:37.101027 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.100999    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/26b203ca-0b58-4389-ac85-8fdc909e11d2-snapshots\") pod \"insights-operator-5885fdb759-mfvfs\" (UID: \"26b203ca-0b58-4389-ac85-8fdc909e11d2\") " pod="openshift-insights/insights-operator-5885fdb759-mfvfs"
Jul 28 19:59:37.101136 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.101046    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/26b203ca-0b58-4389-ac85-8fdc909e11d2-serving-cert\") pod \"insights-operator-5885fdb759-mfvfs\" (UID: \"26b203ca-0b58-4389-ac85-8fdc909e11d2\") " pod="openshift-insights/insights-operator-5885fdb759-mfvfs"
Jul 28 19:59:37.101136 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.101107    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/dde4c1a8-8229-4419-87b6-d8e39a0827c4-config\") pod \"console-operator-584dcb6b54-7vcr8\" (UID: \"dde4c1a8-8229-4419-87b6-d8e39a0827c4\") " pod="openshift-console-operator/console-operator-584dcb6b54-7vcr8"
Jul 28 19:59:37.101226 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.101151    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dde4c1a8-8229-4419-87b6-d8e39a0827c4-trusted-ca\") pod \"console-operator-584dcb6b54-7vcr8\" (UID: \"dde4c1a8-8229-4419-87b6-d8e39a0827c4\") " pod="openshift-console-operator/console-operator-584dcb6b54-7vcr8"
Jul 28 19:59:37.101226 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.101181    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6l9xc\" (UniqueName: \"kubernetes.io/projected/dde4c1a8-8229-4419-87b6-d8e39a0827c4-kube-api-access-6l9xc\") pod \"console-operator-584dcb6b54-7vcr8\" (UID: \"dde4c1a8-8229-4419-87b6-d8e39a0827c4\") " pod="openshift-console-operator/console-operator-584dcb6b54-7vcr8"
Jul 28 19:59:37.101226 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.101220    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8f22d663-ad11-402a-a2a6-2f66e778b4ff-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-k9b95\" (UID: \"8f22d663-ad11-402a-a2a6-2f66e778b4ff\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-k9b95"
Jul 28 19:59:37.101365 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.101249    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2d7a85b0-e81f-4686-8889-5096efe82a1a-serving-cert\") pod \"service-ca-operator-5d6c75f84f-xjfzb\" (UID: \"2d7a85b0-e81f-4686-8889-5096efe82a1a\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-xjfzb"
Jul 28 19:59:37.101365 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.101284    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26b203ca-0b58-4389-ac85-8fdc909e11d2-service-ca-bundle\") pod \"insights-operator-5885fdb759-mfvfs\" (UID: \"26b203ca-0b58-4389-ac85-8fdc909e11d2\") " pod="openshift-insights/insights-operator-5885fdb759-mfvfs"
Jul 28 19:59:37.101365 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.101333    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8d2x8\" (UniqueName: \"kubernetes.io/projected/8f22d663-ad11-402a-a2a6-2f66e778b4ff-kube-api-access-8d2x8\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-k9b95\" (UID: \"8f22d663-ad11-402a-a2a6-2f66e778b4ff\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-k9b95"
Jul 28 19:59:37.101497 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.101373    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb24870f-3c94-4836-9e8b-8d6f757c84de-metrics-certs\") pod \"router-default-7c5b95cbd9-xk69c\" (UID: \"bb24870f-3c94-4836-9e8b-8d6f757c84de\") " pod="openshift-ingress/router-default-7c5b95cbd9-xk69c"
Jul 28 19:59:37.101544 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.101503    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcnmg\" (UniqueName: \"kubernetes.io/projected/2d7a85b0-e81f-4686-8889-5096efe82a1a-kube-api-access-xcnmg\") pod \"service-ca-operator-5d6c75f84f-xjfzb\" (UID: \"2d7a85b0-e81f-4686-8889-5096efe82a1a\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-xjfzb"
Jul 28 19:59:37.101594 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.101562    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/dde4c1a8-8229-4419-87b6-d8e39a0827c4-serving-cert\") pod \"console-operator-584dcb6b54-7vcr8\" (UID: \"dde4c1a8-8229-4419-87b6-d8e39a0827c4\") " pod="openshift-console-operator/console-operator-584dcb6b54-7vcr8"
Jul 28 19:59:37.101642 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.101597    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42j2w\" (UniqueName: \"kubernetes.io/projected/bb24870f-3c94-4836-9e8b-8d6f757c84de-kube-api-access-42j2w\") pod \"router-default-7c5b95cbd9-xk69c\" (UID: \"bb24870f-3c94-4836-9e8b-8d6f757c84de\") " pod="openshift-ingress/router-default-7c5b95cbd9-xk69c"
Jul 28 19:59:37.101676 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.101640    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/bb24870f-3c94-4836-9e8b-8d6f757c84de-default-certificate\") pod \"router-default-7c5b95cbd9-xk69c\" (UID: \"bb24870f-3c94-4836-9e8b-8d6f757c84de\") " pod="openshift-ingress/router-default-7c5b95cbd9-xk69c"
Jul 28 19:59:37.101719 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.101694    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb24870f-3c94-4836-9e8b-8d6f757c84de-service-ca-bundle\") pod \"router-default-7c5b95cbd9-xk69c\" (UID: \"bb24870f-3c94-4836-9e8b-8d6f757c84de\") " pod="openshift-ingress/router-default-7c5b95cbd9-xk69c"
Jul 28 19:59:37.101892 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.101876    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/dde4c1a8-8229-4419-87b6-d8e39a0827c4-config\") pod \"console-operator-584dcb6b54-7vcr8\" (UID: \"dde4c1a8-8229-4419-87b6-d8e39a0827c4\") " pod="openshift-console-operator/console-operator-584dcb6b54-7vcr8"
Jul 28 19:59:37.102552 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.102527    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dde4c1a8-8229-4419-87b6-d8e39a0827c4-trusted-ca\") pod \"console-operator-584dcb6b54-7vcr8\" (UID: \"dde4c1a8-8229-4419-87b6-d8e39a0827c4\") " pod="openshift-console-operator/console-operator-584dcb6b54-7vcr8"
Jul 28 19:59:37.105410 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.105388    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/dde4c1a8-8229-4419-87b6-d8e39a0827c4-serving-cert\") pod \"console-operator-584dcb6b54-7vcr8\" (UID: \"dde4c1a8-8229-4419-87b6-d8e39a0827c4\") " pod="openshift-console-operator/console-operator-584dcb6b54-7vcr8"
Jul 28 19:59:37.106718 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.106703    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-p4m9x"]
Jul 28 19:59:37.106864 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.106846    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-5bj5s"
Jul 28 19:59:37.112798 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.110452    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-rjj5t\""
Jul 28 19:59:37.112798 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.110784    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 28 19:59:37.112798 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.111042    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 28 19:59:37.113267 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.113058    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 28 19:59:37.113489 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.113466    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-p4m9x"
Jul 28 19:59:37.114685 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.113742    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 28 19:59:37.116857 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.116831    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 28 19:59:37.117424 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.117353    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-fvtk6\""
Jul 28 19:59:37.117550 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.117501    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 28 19:59:37.117869 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.117700    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 28 19:59:37.119376 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.119352    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6l9xc\" (UniqueName: \"kubernetes.io/projected/dde4c1a8-8229-4419-87b6-d8e39a0827c4-kube-api-access-6l9xc\") pod \"console-operator-584dcb6b54-7vcr8\" (UID: \"dde4c1a8-8229-4419-87b6-d8e39a0827c4\") " pod="openshift-console-operator/console-operator-584dcb6b54-7vcr8"
Jul 28 19:59:37.119486 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.119437    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-5bj5s"]
Jul 28 19:59:37.119954 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.119933    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlkh9\" (UniqueName: \"kubernetes.io/projected/454f1f68-9403-4c21-ba51-55536c8f55c9-kube-api-access-xlkh9\") pod \"volume-data-source-validator-549c6d54cf-bqs2c\" (UID: \"454f1f68-9403-4c21-ba51-55536c8f55c9\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-bqs2c"
Jul 28 19:59:37.120704 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.120688    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-p4m9x"]
Jul 28 19:59:37.202336 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.202264    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42j2w\" (UniqueName: \"kubernetes.io/projected/bb24870f-3c94-4836-9e8b-8d6f757c84de-kube-api-access-42j2w\") pod \"router-default-7c5b95cbd9-xk69c\" (UID: \"bb24870f-3c94-4836-9e8b-8d6f757c84de\") " pod="openshift-ingress/router-default-7c5b95cbd9-xk69c"
Jul 28 19:59:37.202336 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.202306    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/bb24870f-3c94-4836-9e8b-8d6f757c84de-default-certificate\") pod \"router-default-7c5b95cbd9-xk69c\" (UID: \"bb24870f-3c94-4836-9e8b-8d6f757c84de\") " pod="openshift-ingress/router-default-7c5b95cbd9-xk69c"
Jul 28 19:59:37.202336 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.202332    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb24870f-3c94-4836-9e8b-8d6f757c84de-service-ca-bundle\") pod \"router-default-7c5b95cbd9-xk69c\" (UID: \"bb24870f-3c94-4836-9e8b-8d6f757c84de\") " pod="openshift-ingress/router-default-7c5b95cbd9-xk69c"
Jul 28 19:59:37.203152 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.202361    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ct4cz\" (UniqueName: \"kubernetes.io/projected/d0d19ca1-a117-47c3-a19b-eb756ce208cf-kube-api-access-ct4cz\") pod \"network-check-source-7d88cb55d8-72w9l\" (UID: \"d0d19ca1-a117-47c3-a19b-eb756ce208cf\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-72w9l"
Jul 28 19:59:37.203152 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.202391    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/689af28a-582d-4dfe-a7f9-37cad631f34a-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-p4m9x\" (UID: \"689af28a-582d-4dfe-a7f9-37cad631f34a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-p4m9x"
Jul 28 19:59:37.203152 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.202434    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/15900109-73a5-4356-b125-6b73b7743570-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-5bj5s\" (UID: \"15900109-73a5-4356-b125-6b73b7743570\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-5bj5s"
Jul 28 19:59:37.203152 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.202474    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/15900109-73a5-4356-b125-6b73b7743570-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-5bj5s\" (UID: \"15900109-73a5-4356-b125-6b73b7743570\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-5bj5s"
Jul 28 19:59:37.203152 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:37.202517    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bb24870f-3c94-4836-9e8b-8d6f757c84de-service-ca-bundle podName:bb24870f-3c94-4836-9e8b-8d6f757c84de nodeName:}" failed. No retries permitted until 2026-07-28 19:59:37.702494487 +0000 UTC m=+40.485505924 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/bb24870f-3c94-4836-9e8b-8d6f757c84de-service-ca-bundle") pod "router-default-7c5b95cbd9-xk69c" (UID: "bb24870f-3c94-4836-9e8b-8d6f757c84de") : configmap references non-existent config key: service-ca.crt
Jul 28 19:59:37.203152 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.202560    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/26b203ca-0b58-4389-ac85-8fdc909e11d2-tmp\") pod \"insights-operator-5885fdb759-mfvfs\" (UID: \"26b203ca-0b58-4389-ac85-8fdc909e11d2\") " pod="openshift-insights/insights-operator-5885fdb759-mfvfs"
Jul 28 19:59:37.203152 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.202585    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26b203ca-0b58-4389-ac85-8fdc909e11d2-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-mfvfs\" (UID: \"26b203ca-0b58-4389-ac85-8fdc909e11d2\") " pod="openshift-insights/insights-operator-5885fdb759-mfvfs"
Jul 28 19:59:37.203152 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.202602    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdfbx\" (UniqueName: \"kubernetes.io/projected/26b203ca-0b58-4389-ac85-8fdc909e11d2-kube-api-access-sdfbx\") pod \"insights-operator-5885fdb759-mfvfs\" (UID: \"26b203ca-0b58-4389-ac85-8fdc909e11d2\") " pod="openshift-insights/insights-operator-5885fdb759-mfvfs"
Jul 28 19:59:37.203152 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.202635    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8f22d663-ad11-402a-a2a6-2f66e778b4ff-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-k9b95\" (UID: \"8f22d663-ad11-402a-a2a6-2f66e778b4ff\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-k9b95"
Jul 28 19:59:37.203152 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.202673    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/bb24870f-3c94-4836-9e8b-8d6f757c84de-stats-auth\") pod \"router-default-7c5b95cbd9-xk69c\" (UID: \"bb24870f-3c94-4836-9e8b-8d6f757c84de\") " pod="openshift-ingress/router-default-7c5b95cbd9-xk69c"
Jul 28 19:59:37.203152 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.202709    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2d7a85b0-e81f-4686-8889-5096efe82a1a-config\") pod \"service-ca-operator-5d6c75f84f-xjfzb\" (UID: \"2d7a85b0-e81f-4686-8889-5096efe82a1a\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-xjfzb"
Jul 28 19:59:37.203152 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.202742    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/26b203ca-0b58-4389-ac85-8fdc909e11d2-snapshots\") pod \"insights-operator-5885fdb759-mfvfs\" (UID: \"26b203ca-0b58-4389-ac85-8fdc909e11d2\") " pod="openshift-insights/insights-operator-5885fdb759-mfvfs"
Jul 28 19:59:37.203152 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.202787    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2prl\" (UniqueName: \"kubernetes.io/projected/15900109-73a5-4356-b125-6b73b7743570-kube-api-access-n2prl\") pod \"cluster-monitoring-operator-548b489cb8-5bj5s\" (UID: \"15900109-73a5-4356-b125-6b73b7743570\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-5bj5s"
Jul 28 19:59:37.203152 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.202846    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/26b203ca-0b58-4389-ac85-8fdc909e11d2-serving-cert\") pod \"insights-operator-5885fdb759-mfvfs\" (UID: \"26b203ca-0b58-4389-ac85-8fdc909e11d2\") " pod="openshift-insights/insights-operator-5885fdb759-mfvfs"
Jul 28 19:59:37.203826 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.202876    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8f22d663-ad11-402a-a2a6-2f66e778b4ff-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-k9b95\" (UID: \"8f22d663-ad11-402a-a2a6-2f66e778b4ff\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-k9b95"
Jul 28 19:59:37.203826 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.202912    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2d7a85b0-e81f-4686-8889-5096efe82a1a-serving-cert\") pod \"service-ca-operator-5d6c75f84f-xjfzb\" (UID: \"2d7a85b0-e81f-4686-8889-5096efe82a1a\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-xjfzb"
Jul 28 19:59:37.203826 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.202941    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26b203ca-0b58-4389-ac85-8fdc909e11d2-service-ca-bundle\") pod \"insights-operator-5885fdb759-mfvfs\" (UID: \"26b203ca-0b58-4389-ac85-8fdc909e11d2\") " pod="openshift-insights/insights-operator-5885fdb759-mfvfs"
Jul 28 19:59:37.203826 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.202966    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8d2x8\" (UniqueName: \"kubernetes.io/projected/8f22d663-ad11-402a-a2a6-2f66e778b4ff-kube-api-access-8d2x8\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-k9b95\" (UID: \"8f22d663-ad11-402a-a2a6-2f66e778b4ff\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-k9b95"
Jul 28 19:59:37.203826 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.202982    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb24870f-3c94-4836-9e8b-8d6f757c84de-metrics-certs\") pod \"router-default-7c5b95cbd9-xk69c\" (UID: \"bb24870f-3c94-4836-9e8b-8d6f757c84de\") " pod="openshift-ingress/router-default-7c5b95cbd9-xk69c"
Jul 28 19:59:37.203826 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.203001    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2t7jn\" (UniqueName: \"kubernetes.io/projected/689af28a-582d-4dfe-a7f9-37cad631f34a-kube-api-access-2t7jn\") pod \"cluster-samples-operator-6748489656-p4m9x\" (UID: \"689af28a-582d-4dfe-a7f9-37cad631f34a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-p4m9x"
Jul 28 19:59:37.203826 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.203042    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcnmg\" (UniqueName: \"kubernetes.io/projected/2d7a85b0-e81f-4686-8889-5096efe82a1a-kube-api-access-xcnmg\") pod \"service-ca-operator-5d6c75f84f-xjfzb\" (UID: \"2d7a85b0-e81f-4686-8889-5096efe82a1a\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-xjfzb"
Jul 28 19:59:37.203826 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.203243    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/26b203ca-0b58-4389-ac85-8fdc909e11d2-tmp\") pod \"insights-operator-5885fdb759-mfvfs\" (UID: \"26b203ca-0b58-4389-ac85-8fdc909e11d2\") " pod="openshift-insights/insights-operator-5885fdb759-mfvfs"
Jul 28 19:59:37.203826 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.203589    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/26b203ca-0b58-4389-ac85-8fdc909e11d2-snapshots\") pod \"insights-operator-5885fdb759-mfvfs\" (UID: \"26b203ca-0b58-4389-ac85-8fdc909e11d2\") " pod="openshift-insights/insights-operator-5885fdb759-mfvfs"
Jul 28 19:59:37.203826 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:37.203681    2554 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 19:59:37.203826 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:37.203734    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb24870f-3c94-4836-9e8b-8d6f757c84de-metrics-certs podName:bb24870f-3c94-4836-9e8b-8d6f757c84de nodeName:}" failed. No retries permitted until 2026-07-28 19:59:37.703716332 +0000 UTC m=+40.486727760 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bb24870f-3c94-4836-9e8b-8d6f757c84de-metrics-certs") pod "router-default-7c5b95cbd9-xk69c" (UID: "bb24870f-3c94-4836-9e8b-8d6f757c84de") : secret "router-metrics-certs-default" not found
Jul 28 19:59:37.204372 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.203910    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2d7a85b0-e81f-4686-8889-5096efe82a1a-config\") pod \"service-ca-operator-5d6c75f84f-xjfzb\" (UID: \"2d7a85b0-e81f-4686-8889-5096efe82a1a\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-xjfzb"
Jul 28 19:59:37.204372 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.204149    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26b203ca-0b58-4389-ac85-8fdc909e11d2-service-ca-bundle\") pod \"insights-operator-5885fdb759-mfvfs\" (UID: \"26b203ca-0b58-4389-ac85-8fdc909e11d2\") " pod="openshift-insights/insights-operator-5885fdb759-mfvfs"
Jul 28 19:59:37.204445 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.204395    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26b203ca-0b58-4389-ac85-8fdc909e11d2-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-mfvfs\" (UID: \"26b203ca-0b58-4389-ac85-8fdc909e11d2\") " pod="openshift-insights/insights-operator-5885fdb759-mfvfs"
Jul 28 19:59:37.204445 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.204406    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8f22d663-ad11-402a-a2a6-2f66e778b4ff-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-k9b95\" (UID: \"8f22d663-ad11-402a-a2a6-2f66e778b4ff\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-k9b95"
Jul 28 19:59:37.205298 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.205271    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8f22d663-ad11-402a-a2a6-2f66e778b4ff-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-k9b95\" (UID: \"8f22d663-ad11-402a-a2a6-2f66e778b4ff\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-k9b95"
Jul 28 19:59:37.205407 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.205266    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/bb24870f-3c94-4836-9e8b-8d6f757c84de-default-certificate\") pod \"router-default-7c5b95cbd9-xk69c\" (UID: \"bb24870f-3c94-4836-9e8b-8d6f757c84de\") " pod="openshift-ingress/router-default-7c5b95cbd9-xk69c"
Jul 28 19:59:37.205480 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.205463    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2d7a85b0-e81f-4686-8889-5096efe82a1a-serving-cert\") pod \"service-ca-operator-5d6c75f84f-xjfzb\" (UID: \"2d7a85b0-e81f-4686-8889-5096efe82a1a\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-xjfzb"
Jul 28 19:59:37.206108 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.206090    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/bb24870f-3c94-4836-9e8b-8d6f757c84de-stats-auth\") pod \"router-default-7c5b95cbd9-xk69c\" (UID: \"bb24870f-3c94-4836-9e8b-8d6f757c84de\") " pod="openshift-ingress/router-default-7c5b95cbd9-xk69c"
Jul 28 19:59:37.206471 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.206455    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/26b203ca-0b58-4389-ac85-8fdc909e11d2-serving-cert\") pod \"insights-operator-5885fdb759-mfvfs\" (UID: \"26b203ca-0b58-4389-ac85-8fdc909e11d2\") " pod="openshift-insights/insights-operator-5885fdb759-mfvfs"
Jul 28 19:59:37.211450 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.211426    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcnmg\" (UniqueName: \"kubernetes.io/projected/2d7a85b0-e81f-4686-8889-5096efe82a1a-kube-api-access-xcnmg\") pod \"service-ca-operator-5d6c75f84f-xjfzb\" (UID: \"2d7a85b0-e81f-4686-8889-5096efe82a1a\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-xjfzb"
Jul 28 19:59:37.212674 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.212637    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ct4cz\" (UniqueName: \"kubernetes.io/projected/d0d19ca1-a117-47c3-a19b-eb756ce208cf-kube-api-access-ct4cz\") pod \"network-check-source-7d88cb55d8-72w9l\" (UID: \"d0d19ca1-a117-47c3-a19b-eb756ce208cf\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-72w9l"
Jul 28 19:59:37.212787 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.212767    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8d2x8\" (UniqueName: \"kubernetes.io/projected/8f22d663-ad11-402a-a2a6-2f66e778b4ff-kube-api-access-8d2x8\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-k9b95\" (UID: \"8f22d663-ad11-402a-a2a6-2f66e778b4ff\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-k9b95"
Jul 28 19:59:37.213060 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.213040    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdfbx\" (UniqueName: \"kubernetes.io/projected/26b203ca-0b58-4389-ac85-8fdc909e11d2-kube-api-access-sdfbx\") pod \"insights-operator-5885fdb759-mfvfs\" (UID: \"26b203ca-0b58-4389-ac85-8fdc909e11d2\") " pod="openshift-insights/insights-operator-5885fdb759-mfvfs"
Jul 28 19:59:37.213341 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.213323    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42j2w\" (UniqueName: \"kubernetes.io/projected/bb24870f-3c94-4836-9e8b-8d6f757c84de-kube-api-access-42j2w\") pod \"router-default-7c5b95cbd9-xk69c\" (UID: \"bb24870f-3c94-4836-9e8b-8d6f757c84de\") " pod="openshift-ingress/router-default-7c5b95cbd9-xk69c"
Jul 28 19:59:37.226591 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.226570    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-bqs2c"
Jul 28 19:59:37.250415 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.250394    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-7vcr8"
Jul 28 19:59:37.306155 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.304012    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/689af28a-582d-4dfe-a7f9-37cad631f34a-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-p4m9x\" (UID: \"689af28a-582d-4dfe-a7f9-37cad631f34a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-p4m9x"
Jul 28 19:59:37.306155 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.304090    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/15900109-73a5-4356-b125-6b73b7743570-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-5bj5s\" (UID: \"15900109-73a5-4356-b125-6b73b7743570\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-5bj5s"
Jul 28 19:59:37.306155 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.304122    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/15900109-73a5-4356-b125-6b73b7743570-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-5bj5s\" (UID: \"15900109-73a5-4356-b125-6b73b7743570\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-5bj5s"
Jul 28 19:59:37.306155 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.304210    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2prl\" (UniqueName: \"kubernetes.io/projected/15900109-73a5-4356-b125-6b73b7743570-kube-api-access-n2prl\") pod \"cluster-monitoring-operator-548b489cb8-5bj5s\" (UID: \"15900109-73a5-4356-b125-6b73b7743570\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-5bj5s"
Jul 28 19:59:37.306155 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.304283    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2t7jn\" (UniqueName: \"kubernetes.io/projected/689af28a-582d-4dfe-a7f9-37cad631f34a-kube-api-access-2t7jn\") pod \"cluster-samples-operator-6748489656-p4m9x\" (UID: \"689af28a-582d-4dfe-a7f9-37cad631f34a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-p4m9x"
Jul 28 19:59:37.306155 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:37.304766    2554 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 19:59:37.306155 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:37.304832    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/689af28a-582d-4dfe-a7f9-37cad631f34a-samples-operator-tls podName:689af28a-582d-4dfe-a7f9-37cad631f34a nodeName:}" failed. No retries permitted until 2026-07-28 19:59:37.80481187 +0000 UTC m=+40.587823310 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/689af28a-582d-4dfe-a7f9-37cad631f34a-samples-operator-tls") pod "cluster-samples-operator-6748489656-p4m9x" (UID: "689af28a-582d-4dfe-a7f9-37cad631f34a") : secret "samples-operator-tls" not found
Jul 28 19:59:37.306155 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:37.305024    2554 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 19:59:37.306155 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:37.305060    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15900109-73a5-4356-b125-6b73b7743570-cluster-monitoring-operator-tls podName:15900109-73a5-4356-b125-6b73b7743570 nodeName:}" failed. No retries permitted until 2026-07-28 19:59:37.805048283 +0000 UTC m=+40.588059723 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/15900109-73a5-4356-b125-6b73b7743570-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-5bj5s" (UID: "15900109-73a5-4356-b125-6b73b7743570") : secret "cluster-monitoring-operator-tls" not found
Jul 28 19:59:37.306155 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.305841    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/15900109-73a5-4356-b125-6b73b7743570-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-5bj5s\" (UID: \"15900109-73a5-4356-b125-6b73b7743570\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-5bj5s"
Jul 28 19:59:37.318091 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.315733    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-72w9l"
Jul 28 19:59:37.324472 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.324382    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-xjfzb"
Jul 28 19:59:37.325118 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.325091    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2t7jn\" (UniqueName: \"kubernetes.io/projected/689af28a-582d-4dfe-a7f9-37cad631f34a-kube-api-access-2t7jn\") pod \"cluster-samples-operator-6748489656-p4m9x\" (UID: \"689af28a-582d-4dfe-a7f9-37cad631f34a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-p4m9x"
Jul 28 19:59:37.328307 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.328255    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2prl\" (UniqueName: \"kubernetes.io/projected/15900109-73a5-4356-b125-6b73b7743570-kube-api-access-n2prl\") pod \"cluster-monitoring-operator-548b489cb8-5bj5s\" (UID: \"15900109-73a5-4356-b125-6b73b7743570\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-5bj5s"
Jul 28 19:59:37.335326 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.335278    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-k9b95"
Jul 28 19:59:37.369323 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.369299    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-mfvfs"
Jul 28 19:59:37.400938 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.400851    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-bqs2c"]
Jul 28 19:59:37.404579 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:59:37.404546    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod454f1f68_9403_4c21_ba51_55536c8f55c9.slice/crio-5829eb623608ca854fbf32a4dc328b6a5488c8d57d0ba952270ced4d4b1cd027 WatchSource:0}: Error finding container 5829eb623608ca854fbf32a4dc328b6a5488c8d57d0ba952270ced4d4b1cd027: Status 404 returned error can't find the container with id 5829eb623608ca854fbf32a4dc328b6a5488c8d57d0ba952270ced4d4b1cd027
Jul 28 19:59:37.420059 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.419985    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-7vcr8"]
Jul 28 19:59:37.503441 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.503370    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-72w9l"]
Jul 28 19:59:37.510505 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:59:37.510465    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0d19ca1_a117_47c3_a19b_eb756ce208cf.slice/crio-ad9aa8106b0ea4605c32933b0464e4b37effe1a1774ba69507f77a999c3061b4 WatchSource:0}: Error finding container ad9aa8106b0ea4605c32933b0464e4b37effe1a1774ba69507f77a999c3061b4: Status 404 returned error can't find the container with id ad9aa8106b0ea4605c32933b0464e4b37effe1a1774ba69507f77a999c3061b4
Jul 28 19:59:37.528791 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.528635    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-xjfzb"]
Jul 28 19:59:37.532575 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:59:37.532546    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d7a85b0_e81f_4686_8889_5096efe82a1a.slice/crio-73a1c8b992626ba80f7b17066f68c9c51adae121e7db0de864e74181b0489b62 WatchSource:0}: Error finding container 73a1c8b992626ba80f7b17066f68c9c51adae121e7db0de864e74181b0489b62: Status 404 returned error can't find the container with id 73a1c8b992626ba80f7b17066f68c9c51adae121e7db0de864e74181b0489b62
Jul 28 19:59:37.562241 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.562209    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-k9b95"]
Jul 28 19:59:37.565012 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:59:37.564988    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f22d663_ad11_402a_a2a6_2f66e778b4ff.slice/crio-54b06458e784a20ce242e3d86489b8acf8710135ee4c5b99c490ebb4b7231ba2 WatchSource:0}: Error finding container 54b06458e784a20ce242e3d86489b8acf8710135ee4c5b99c490ebb4b7231ba2: Status 404 returned error can't find the container with id 54b06458e784a20ce242e3d86489b8acf8710135ee4c5b99c490ebb4b7231ba2
Jul 28 19:59:37.577724 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.577702    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-mfvfs"]
Jul 28 19:59:37.580162 ip-10-0-139-54 kubenswrapper[2554]: W0728 19:59:37.580128    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26b203ca_0b58_4389_ac85_8fdc909e11d2.slice/crio-dd08b8955ab8bcc2f32873da5477d3f7b34d858634eb66f07a6e9909c9ab52fc WatchSource:0}: Error finding container dd08b8955ab8bcc2f32873da5477d3f7b34d858634eb66f07a6e9909c9ab52fc: Status 404 returned error can't find the container with id dd08b8955ab8bcc2f32873da5477d3f7b34d858634eb66f07a6e9909c9ab52fc
Jul 28 19:59:37.708319 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.708286    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb24870f-3c94-4836-9e8b-8d6f757c84de-metrics-certs\") pod \"router-default-7c5b95cbd9-xk69c\" (UID: \"bb24870f-3c94-4836-9e8b-8d6f757c84de\") " pod="openshift-ingress/router-default-7c5b95cbd9-xk69c"
Jul 28 19:59:37.708457 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.708332    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb24870f-3c94-4836-9e8b-8d6f757c84de-service-ca-bundle\") pod \"router-default-7c5b95cbd9-xk69c\" (UID: \"bb24870f-3c94-4836-9e8b-8d6f757c84de\") " pod="openshift-ingress/router-default-7c5b95cbd9-xk69c"
Jul 28 19:59:37.708457 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:37.708443    2554 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 19:59:37.708537 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:37.708514    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb24870f-3c94-4836-9e8b-8d6f757c84de-metrics-certs podName:bb24870f-3c94-4836-9e8b-8d6f757c84de nodeName:}" failed. No retries permitted until 2026-07-28 19:59:38.70849421 +0000 UTC m=+41.491505632 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bb24870f-3c94-4836-9e8b-8d6f757c84de-metrics-certs") pod "router-default-7c5b95cbd9-xk69c" (UID: "bb24870f-3c94-4836-9e8b-8d6f757c84de") : secret "router-metrics-certs-default" not found
Jul 28 19:59:37.708615 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:37.708602    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bb24870f-3c94-4836-9e8b-8d6f757c84de-service-ca-bundle podName:bb24870f-3c94-4836-9e8b-8d6f757c84de nodeName:}" failed. No retries permitted until 2026-07-28 19:59:38.70859098 +0000 UTC m=+41.491602400 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/bb24870f-3c94-4836-9e8b-8d6f757c84de-service-ca-bundle") pod "router-default-7c5b95cbd9-xk69c" (UID: "bb24870f-3c94-4836-9e8b-8d6f757c84de") : configmap references non-existent config key: service-ca.crt
Jul 28 19:59:37.809188 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.809153    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/689af28a-582d-4dfe-a7f9-37cad631f34a-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-p4m9x\" (UID: \"689af28a-582d-4dfe-a7f9-37cad631f34a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-p4m9x"
Jul 28 19:59:37.809335 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:37.809197    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/15900109-73a5-4356-b125-6b73b7743570-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-5bj5s\" (UID: \"15900109-73a5-4356-b125-6b73b7743570\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-5bj5s"
Jul 28 19:59:37.809335 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:37.809293    2554 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 19:59:37.809335 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:37.809314    2554 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 19:59:37.809444 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:37.809346    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15900109-73a5-4356-b125-6b73b7743570-cluster-monitoring-operator-tls podName:15900109-73a5-4356-b125-6b73b7743570 nodeName:}" failed. No retries permitted until 2026-07-28 19:59:38.809331753 +0000 UTC m=+41.592343173 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/15900109-73a5-4356-b125-6b73b7743570-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-5bj5s" (UID: "15900109-73a5-4356-b125-6b73b7743570") : secret "cluster-monitoring-operator-tls" not found
Jul 28 19:59:37.809444 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:37.809373    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/689af28a-582d-4dfe-a7f9-37cad631f34a-samples-operator-tls podName:689af28a-582d-4dfe-a7f9-37cad631f34a nodeName:}" failed. No retries permitted until 2026-07-28 19:59:38.809361263 +0000 UTC m=+41.592372683 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/689af28a-582d-4dfe-a7f9-37cad631f34a-samples-operator-tls") pod "cluster-samples-operator-6748489656-p4m9x" (UID: "689af28a-582d-4dfe-a7f9-37cad631f34a") : secret "samples-operator-tls" not found
Jul 28 19:59:38.145418 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:38.145022    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-k9b95" event={"ID":"8f22d663-ad11-402a-a2a6-2f66e778b4ff","Type":"ContainerStarted","Data":"54b06458e784a20ce242e3d86489b8acf8710135ee4c5b99c490ebb4b7231ba2"}
Jul 28 19:59:38.146430 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:38.146399    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-mfvfs" event={"ID":"26b203ca-0b58-4389-ac85-8fdc909e11d2","Type":"ContainerStarted","Data":"dd08b8955ab8bcc2f32873da5477d3f7b34d858634eb66f07a6e9909c9ab52fc"}
Jul 28 19:59:38.148042 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:38.148010    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-xjfzb" event={"ID":"2d7a85b0-e81f-4686-8889-5096efe82a1a","Type":"ContainerStarted","Data":"73a1c8b992626ba80f7b17066f68c9c51adae121e7db0de864e74181b0489b62"}
Jul 28 19:59:38.149366 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:38.149312    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-72w9l" event={"ID":"d0d19ca1-a117-47c3-a19b-eb756ce208cf","Type":"ContainerStarted","Data":"ad9aa8106b0ea4605c32933b0464e4b37effe1a1774ba69507f77a999c3061b4"}
Jul 28 19:59:38.150698 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:38.150646    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-7vcr8" event={"ID":"dde4c1a8-8229-4419-87b6-d8e39a0827c4","Type":"ContainerStarted","Data":"aa4ae820a6a5b9f84e9bc174eadc9802dd423b0fd2674a55c4f4a9002e565bdb"}
Jul 28 19:59:38.151937 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:38.151915    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-bqs2c" event={"ID":"454f1f68-9403-4c21-ba51-55536c8f55c9","Type":"ContainerStarted","Data":"5829eb623608ca854fbf32a4dc328b6a5488c8d57d0ba952270ced4d4b1cd027"}
Jul 28 19:59:38.719611 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:38.719567    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb24870f-3c94-4836-9e8b-8d6f757c84de-metrics-certs\") pod \"router-default-7c5b95cbd9-xk69c\" (UID: \"bb24870f-3c94-4836-9e8b-8d6f757c84de\") " pod="openshift-ingress/router-default-7c5b95cbd9-xk69c"
Jul 28 19:59:38.720120 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:38.719655    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb24870f-3c94-4836-9e8b-8d6f757c84de-service-ca-bundle\") pod \"router-default-7c5b95cbd9-xk69c\" (UID: \"bb24870f-3c94-4836-9e8b-8d6f757c84de\") " pod="openshift-ingress/router-default-7c5b95cbd9-xk69c"
Jul 28 19:59:38.720120 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:38.719760    2554 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 19:59:38.720120 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:38.719825    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bb24870f-3c94-4836-9e8b-8d6f757c84de-service-ca-bundle podName:bb24870f-3c94-4836-9e8b-8d6f757c84de nodeName:}" failed. No retries permitted until 2026-07-28 19:59:40.719805849 +0000 UTC m=+43.502817272 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/bb24870f-3c94-4836-9e8b-8d6f757c84de-service-ca-bundle") pod "router-default-7c5b95cbd9-xk69c" (UID: "bb24870f-3c94-4836-9e8b-8d6f757c84de") : configmap references non-existent config key: service-ca.crt
Jul 28 19:59:38.720120 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:38.719848    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb24870f-3c94-4836-9e8b-8d6f757c84de-metrics-certs podName:bb24870f-3c94-4836-9e8b-8d6f757c84de nodeName:}" failed. No retries permitted until 2026-07-28 19:59:40.71983663 +0000 UTC m=+43.502848050 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bb24870f-3c94-4836-9e8b-8d6f757c84de-metrics-certs") pod "router-default-7c5b95cbd9-xk69c" (UID: "bb24870f-3c94-4836-9e8b-8d6f757c84de") : secret "router-metrics-certs-default" not found
Jul 28 19:59:38.821158 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:38.820325    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/689af28a-582d-4dfe-a7f9-37cad631f34a-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-p4m9x\" (UID: \"689af28a-582d-4dfe-a7f9-37cad631f34a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-p4m9x"
Jul 28 19:59:38.821158 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:38.820389    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/15900109-73a5-4356-b125-6b73b7743570-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-5bj5s\" (UID: \"15900109-73a5-4356-b125-6b73b7743570\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-5bj5s"
Jul 28 19:59:38.821158 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:38.820553    2554 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 19:59:38.821158 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:38.820616    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15900109-73a5-4356-b125-6b73b7743570-cluster-monitoring-operator-tls podName:15900109-73a5-4356-b125-6b73b7743570 nodeName:}" failed. No retries permitted until 2026-07-28 19:59:40.82059729 +0000 UTC m=+43.603608726 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/15900109-73a5-4356-b125-6b73b7743570-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-5bj5s" (UID: "15900109-73a5-4356-b125-6b73b7743570") : secret "cluster-monitoring-operator-tls" not found
Jul 28 19:59:38.821158 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:38.821039    2554 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 19:59:38.821158 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:38.821110    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/689af28a-582d-4dfe-a7f9-37cad631f34a-samples-operator-tls podName:689af28a-582d-4dfe-a7f9-37cad631f34a nodeName:}" failed. No retries permitted until 2026-07-28 19:59:40.821093847 +0000 UTC m=+43.604105290 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/689af28a-582d-4dfe-a7f9-37cad631f34a-samples-operator-tls") pod "cluster-samples-operator-6748489656-p4m9x" (UID: "689af28a-582d-4dfe-a7f9-37cad631f34a") : secret "samples-operator-tls" not found
Jul 28 19:59:39.426752 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:39.426102    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0dda64d8-2cc8-4164-afc1-303f060864a8-registry-tls\") pod \"image-registry-7bcc689c88-gmt7f\" (UID: \"0dda64d8-2cc8-4164-afc1-303f060864a8\") " pod="openshift-image-registry/image-registry-7bcc689c88-gmt7f"
Jul 28 19:59:39.426752 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:39.426222    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2b7b89ba-b64b-454a-b032-34757974c00a-metrics-tls\") pod \"dns-default-qs4pv\" (UID: \"2b7b89ba-b64b-454a-b032-34757974c00a\") " pod="openshift-dns/dns-default-qs4pv"
Jul 28 19:59:39.426752 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:39.426395    2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 19:59:39.426752 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:39.426458    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b7b89ba-b64b-454a-b032-34757974c00a-metrics-tls podName:2b7b89ba-b64b-454a-b032-34757974c00a nodeName:}" failed. No retries permitted until 2026-07-28 19:59:47.426440446 +0000 UTC m=+50.209451889 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2b7b89ba-b64b-454a-b032-34757974c00a-metrics-tls") pod "dns-default-qs4pv" (UID: "2b7b89ba-b64b-454a-b032-34757974c00a") : secret "dns-default-metrics-tls" not found
Jul 28 19:59:39.426752 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:39.426532    2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 19:59:39.426752 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:39.426544    2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7bcc689c88-gmt7f: secret "image-registry-tls" not found
Jul 28 19:59:39.426752 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:39.426587    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0dda64d8-2cc8-4164-afc1-303f060864a8-registry-tls podName:0dda64d8-2cc8-4164-afc1-303f060864a8 nodeName:}" failed. No retries permitted until 2026-07-28 19:59:47.426575735 +0000 UTC m=+50.209587160 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0dda64d8-2cc8-4164-afc1-303f060864a8-registry-tls") pod "image-registry-7bcc689c88-gmt7f" (UID: "0dda64d8-2cc8-4164-afc1-303f060864a8") : secret "image-registry-tls" not found
Jul 28 19:59:40.738915 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:40.738868    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb24870f-3c94-4836-9e8b-8d6f757c84de-metrics-certs\") pod \"router-default-7c5b95cbd9-xk69c\" (UID: \"bb24870f-3c94-4836-9e8b-8d6f757c84de\") " pod="openshift-ingress/router-default-7c5b95cbd9-xk69c"
Jul 28 19:59:40.739423 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:40.738957    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb24870f-3c94-4836-9e8b-8d6f757c84de-service-ca-bundle\") pod \"router-default-7c5b95cbd9-xk69c\" (UID: \"bb24870f-3c94-4836-9e8b-8d6f757c84de\") " pod="openshift-ingress/router-default-7c5b95cbd9-xk69c"
Jul 28 19:59:40.739423 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:40.739051    2554 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 19:59:40.739423 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:40.739113    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bb24870f-3c94-4836-9e8b-8d6f757c84de-service-ca-bundle podName:bb24870f-3c94-4836-9e8b-8d6f757c84de nodeName:}" failed. No retries permitted until 2026-07-28 19:59:44.739087788 +0000 UTC m=+47.522099228 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/bb24870f-3c94-4836-9e8b-8d6f757c84de-service-ca-bundle") pod "router-default-7c5b95cbd9-xk69c" (UID: "bb24870f-3c94-4836-9e8b-8d6f757c84de") : configmap references non-existent config key: service-ca.crt
Jul 28 19:59:40.739423 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:40.739141    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb24870f-3c94-4836-9e8b-8d6f757c84de-metrics-certs podName:bb24870f-3c94-4836-9e8b-8d6f757c84de nodeName:}" failed. No retries permitted until 2026-07-28 19:59:44.739130963 +0000 UTC m=+47.522142383 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bb24870f-3c94-4836-9e8b-8d6f757c84de-metrics-certs") pod "router-default-7c5b95cbd9-xk69c" (UID: "bb24870f-3c94-4836-9e8b-8d6f757c84de") : secret "router-metrics-certs-default" not found
Jul 28 19:59:40.840366 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:40.840326    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/689af28a-582d-4dfe-a7f9-37cad631f34a-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-p4m9x\" (UID: \"689af28a-582d-4dfe-a7f9-37cad631f34a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-p4m9x"
Jul 28 19:59:40.840366 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:40.840373    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/15900109-73a5-4356-b125-6b73b7743570-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-5bj5s\" (UID: \"15900109-73a5-4356-b125-6b73b7743570\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-5bj5s"
Jul 28 19:59:40.840548 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:40.840479    2554 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 19:59:40.840548 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:40.840481    2554 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 19:59:40.840548 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:40.840544    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/689af28a-582d-4dfe-a7f9-37cad631f34a-samples-operator-tls podName:689af28a-582d-4dfe-a7f9-37cad631f34a nodeName:}" failed. No retries permitted until 2026-07-28 19:59:44.840529932 +0000 UTC m=+47.623541353 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/689af28a-582d-4dfe-a7f9-37cad631f34a-samples-operator-tls") pod "cluster-samples-operator-6748489656-p4m9x" (UID: "689af28a-582d-4dfe-a7f9-37cad631f34a") : secret "samples-operator-tls" not found
Jul 28 19:59:40.840645 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:40.840562    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15900109-73a5-4356-b125-6b73b7743570-cluster-monitoring-operator-tls podName:15900109-73a5-4356-b125-6b73b7743570 nodeName:}" failed. No retries permitted until 2026-07-28 19:59:44.840552558 +0000 UTC m=+47.623563978 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/15900109-73a5-4356-b125-6b73b7743570-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-5bj5s" (UID: "15900109-73a5-4356-b125-6b73b7743570") : secret "cluster-monitoring-operator-tls" not found
Jul 28 19:59:44.173277 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:44.173243    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-72w9l" event={"ID":"d0d19ca1-a117-47c3-a19b-eb756ce208cf","Type":"ContainerStarted","Data":"378a3d49a90ce8ea9cf62c3d2bf09c5db0c81e1521d95b1a4f214e1d5078fdfb"}
Jul 28 19:59:44.175437 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:44.175415    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-7vcr8" event={"ID":"dde4c1a8-8229-4419-87b6-d8e39a0827c4","Type":"ContainerStarted","Data":"cbc995e403bcc5f23ef04446957f647cb49e3dcd84afe82102f3ec57bb51ea05"}
Jul 28 19:59:44.176065 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:44.175867    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-7vcr8"
Jul 28 19:59:44.177089 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:44.176990    2554 patch_prober.go:28] interesting pod/console-operator-584dcb6b54-7vcr8 container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.10:8443/readyz\": dial tcp 10.132.0.10:8443: connect: connection refused" start-of-body=
Jul 28 19:59:44.177089 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:44.177024    2554 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-584dcb6b54-7vcr8" podUID="dde4c1a8-8229-4419-87b6-d8e39a0827c4" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.10:8443/readyz\": dial tcp 10.132.0.10:8443: connect: connection refused"
Jul 28 19:59:44.177205 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:44.177136    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-bqs2c" event={"ID":"454f1f68-9403-4c21-ba51-55536c8f55c9","Type":"ContainerStarted","Data":"c475408c6b4d0866935795e800045457fc6eb953ad1b57763851ba823ddb1172"}
Jul 28 19:59:44.179973 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:44.179908    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-k9b95" event={"ID":"8f22d663-ad11-402a-a2a6-2f66e778b4ff","Type":"ContainerStarted","Data":"49de1ce79e48987391516775f94530bd30c2b97c2f0eb13e95972065395bc557"}
Jul 28 19:59:44.190706 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:44.190644    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-72w9l" podStartSLOduration=1.7588598260000001 podStartE2EDuration="8.190626426s" totalImagesPullingTime="6.4317666s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:59:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:59:44.190282385 +0000 UTC m=+46.973293827" watchObservedRunningTime="2026-07-28 19:59:44.190626426 +0000 UTC m=+46.973637849"
Jul 28 19:59:44.207602 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:44.207466    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-bqs2c" podStartSLOduration=1.922196339 podStartE2EDuration="8.207446016s" totalImagesPullingTime="6.285249677s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:59:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:59:44.207294527 +0000 UTC m=+46.990305993" watchObservedRunningTime="2026-07-28 19:59:44.207446016 +0000 UTC m=+46.990457461"
Jul 28 19:59:44.244237 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:44.243849    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-7vcr8" podStartSLOduration=1.733171107 podStartE2EDuration="8.243827128s" totalImagesPullingTime="6.510656021s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:59:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:59:44.243569965 +0000 UTC m=+47.026581433" watchObservedRunningTime="2026-07-28 19:59:44.243827128 +0000 UTC m=+47.026838570"
Jul 28 19:59:44.245045 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:44.245001    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-k9b95" podStartSLOduration=1.866990103 podStartE2EDuration="8.244988435s" totalImagesPullingTime="6.377998332s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:59:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:59:44.223241215 +0000 UTC m=+47.006252658" watchObservedRunningTime="2026-07-28 19:59:44.244988435 +0000 UTC m=+47.027999880"
Jul 28 19:59:44.781040 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:44.781005    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb24870f-3c94-4836-9e8b-8d6f757c84de-metrics-certs\") pod \"router-default-7c5b95cbd9-xk69c\" (UID: \"bb24870f-3c94-4836-9e8b-8d6f757c84de\") " pod="openshift-ingress/router-default-7c5b95cbd9-xk69c"
Jul 28 19:59:44.781233 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:44.781057    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb24870f-3c94-4836-9e8b-8d6f757c84de-service-ca-bundle\") pod \"router-default-7c5b95cbd9-xk69c\" (UID: \"bb24870f-3c94-4836-9e8b-8d6f757c84de\") " pod="openshift-ingress/router-default-7c5b95cbd9-xk69c"
Jul 28 19:59:44.781233 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:44.781189    2554 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 19:59:44.781354 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:44.781255    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb24870f-3c94-4836-9e8b-8d6f757c84de-metrics-certs podName:bb24870f-3c94-4836-9e8b-8d6f757c84de nodeName:}" failed. No retries permitted until 2026-07-28 19:59:52.781234876 +0000 UTC m=+55.564246316 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bb24870f-3c94-4836-9e8b-8d6f757c84de-metrics-certs") pod "router-default-7c5b95cbd9-xk69c" (UID: "bb24870f-3c94-4836-9e8b-8d6f757c84de") : secret "router-metrics-certs-default" not found
Jul 28 19:59:44.781354 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:44.781286    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bb24870f-3c94-4836-9e8b-8d6f757c84de-service-ca-bundle podName:bb24870f-3c94-4836-9e8b-8d6f757c84de nodeName:}" failed. No retries permitted until 2026-07-28 19:59:52.78127566 +0000 UTC m=+55.564287299 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/bb24870f-3c94-4836-9e8b-8d6f757c84de-service-ca-bundle") pod "router-default-7c5b95cbd9-xk69c" (UID: "bb24870f-3c94-4836-9e8b-8d6f757c84de") : configmap references non-existent config key: service-ca.crt
Jul 28 19:59:44.882522 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:44.882475    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/689af28a-582d-4dfe-a7f9-37cad631f34a-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-p4m9x\" (UID: \"689af28a-582d-4dfe-a7f9-37cad631f34a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-p4m9x"
Jul 28 19:59:44.882701 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:44.882541    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/15900109-73a5-4356-b125-6b73b7743570-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-5bj5s\" (UID: \"15900109-73a5-4356-b125-6b73b7743570\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-5bj5s"
Jul 28 19:59:44.882701 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:44.882667    2554 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 19:59:44.882846 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:44.882715    2554 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 19:59:44.882846 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:44.882750    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/689af28a-582d-4dfe-a7f9-37cad631f34a-samples-operator-tls podName:689af28a-582d-4dfe-a7f9-37cad631f34a nodeName:}" failed. No retries permitted until 2026-07-28 19:59:52.882728794 +0000 UTC m=+55.665740217 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/689af28a-582d-4dfe-a7f9-37cad631f34a-samples-operator-tls") pod "cluster-samples-operator-6748489656-p4m9x" (UID: "689af28a-582d-4dfe-a7f9-37cad631f34a") : secret "samples-operator-tls" not found
Jul 28 19:59:44.882846 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:44.882784    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15900109-73a5-4356-b125-6b73b7743570-cluster-monitoring-operator-tls podName:15900109-73a5-4356-b125-6b73b7743570 nodeName:}" failed. No retries permitted until 2026-07-28 19:59:52.882767558 +0000 UTC m=+55.665778985 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/15900109-73a5-4356-b125-6b73b7743570-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-5bj5s" (UID: "15900109-73a5-4356-b125-6b73b7743570") : secret "cluster-monitoring-operator-tls" not found
Jul 28 19:59:45.186065 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:45.186028    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-mfvfs" event={"ID":"26b203ca-0b58-4389-ac85-8fdc909e11d2","Type":"ContainerStarted","Data":"56554e28cf207950fdb3c05131dc9c0ca57599719f163ea72fadec35ffe84551"}
Jul 28 19:59:45.187881 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:45.187821    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-xjfzb" event={"ID":"2d7a85b0-e81f-4686-8889-5096efe82a1a","Type":"ContainerStarted","Data":"2c90070ca04436e38308b48a03a960139652b828a6a0916588f7bbcee7fcd86a"}
Jul 28 19:59:45.189621 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:45.189593    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-7vcr8_dde4c1a8-8229-4419-87b6-d8e39a0827c4/console-operator/0.log"
Jul 28 19:59:45.189756 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:45.189633    2554 generic.go:356] "Generic (PLEG): container finished" podID="dde4c1a8-8229-4419-87b6-d8e39a0827c4" containerID="cbc995e403bcc5f23ef04446957f647cb49e3dcd84afe82102f3ec57bb51ea05" exitCode=255
Jul 28 19:59:45.189756 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:45.189729    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-7vcr8" event={"ID":"dde4c1a8-8229-4419-87b6-d8e39a0827c4","Type":"ContainerDied","Data":"cbc995e403bcc5f23ef04446957f647cb49e3dcd84afe82102f3ec57bb51ea05"}
Jul 28 19:59:45.190012 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:45.189993    2554 scope.go:117] "RemoveContainer" containerID="cbc995e403bcc5f23ef04446957f647cb49e3dcd84afe82102f3ec57bb51ea05"
Jul 28 19:59:45.203956 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:45.203895    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-mfvfs" podStartSLOduration=2.8349798059999998 podStartE2EDuration="9.203879367s" totalImagesPullingTime="6.368899561s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:59:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:59:45.20275124 +0000 UTC m=+47.985762683" watchObservedRunningTime="2026-07-28 19:59:45.203879367 +0000 UTC m=+47.986890809"
Jul 28 19:59:45.255483 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:45.255434    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-xjfzb" podStartSLOduration=2.845401122 podStartE2EDuration="9.255417675s" totalImagesPullingTime="6.410016553s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:59:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:59:45.253673673 +0000 UTC m=+48.036685212" watchObservedRunningTime="2026-07-28 19:59:45.255417675 +0000 UTC m=+48.038429117"
Jul 28 19:59:45.338896 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:45.337546    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-trhgs"]
Jul 28 19:59:45.359101 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:45.356193    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-trhgs"
Jul 28 19:59:45.359101 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:45.356986    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-trhgs"]
Jul 28 19:59:45.359435 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:45.359413    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 28 19:59:45.359620 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:45.359598    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 28 19:59:45.359903 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:45.359886    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-mrvdk\""
Jul 28 19:59:45.386952 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:45.386918    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/be39a539-0c2c-4e08-9d85-c5ade81491a4-data-volume\") pod \"insights-runtime-extractor-trhgs\" (UID: \"be39a539-0c2c-4e08-9d85-c5ade81491a4\") " pod="openshift-insights/insights-runtime-extractor-trhgs"
Jul 28 19:59:45.387121 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:45.386963    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qm8n\" (UniqueName: \"kubernetes.io/projected/be39a539-0c2c-4e08-9d85-c5ade81491a4-kube-api-access-7qm8n\") pod \"insights-runtime-extractor-trhgs\" (UID: \"be39a539-0c2c-4e08-9d85-c5ade81491a4\") " pod="openshift-insights/insights-runtime-extractor-trhgs"
Jul 28 19:59:45.387121 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:45.387089    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/be39a539-0c2c-4e08-9d85-c5ade81491a4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-trhgs\" (UID: \"be39a539-0c2c-4e08-9d85-c5ade81491a4\") " pod="openshift-insights/insights-runtime-extractor-trhgs"
Jul 28 19:59:45.387261 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:45.387211    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/be39a539-0c2c-4e08-9d85-c5ade81491a4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-trhgs\" (UID: \"be39a539-0c2c-4e08-9d85-c5ade81491a4\") " pod="openshift-insights/insights-runtime-extractor-trhgs"
Jul 28 19:59:45.387315 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:45.387303    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/be39a539-0c2c-4e08-9d85-c5ade81491a4-crio-socket\") pod \"insights-runtime-extractor-trhgs\" (UID: \"be39a539-0c2c-4e08-9d85-c5ade81491a4\") " pod="openshift-insights/insights-runtime-extractor-trhgs"
Jul 28 19:59:45.488771 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:45.488662    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/be39a539-0c2c-4e08-9d85-c5ade81491a4-crio-socket\") pod \"insights-runtime-extractor-trhgs\" (UID: \"be39a539-0c2c-4e08-9d85-c5ade81491a4\") " pod="openshift-insights/insights-runtime-extractor-trhgs"
Jul 28 19:59:45.488771 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:45.488730    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/be39a539-0c2c-4e08-9d85-c5ade81491a4-data-volume\") pod \"insights-runtime-extractor-trhgs\" (UID: \"be39a539-0c2c-4e08-9d85-c5ade81491a4\") " pod="openshift-insights/insights-runtime-extractor-trhgs"
Jul 28 19:59:45.488771 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:45.488769    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qm8n\" (UniqueName: \"kubernetes.io/projected/be39a539-0c2c-4e08-9d85-c5ade81491a4-kube-api-access-7qm8n\") pod \"insights-runtime-extractor-trhgs\" (UID: \"be39a539-0c2c-4e08-9d85-c5ade81491a4\") " pod="openshift-insights/insights-runtime-extractor-trhgs"
Jul 28 19:59:45.489042 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:45.488833    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/be39a539-0c2c-4e08-9d85-c5ade81491a4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-trhgs\" (UID: \"be39a539-0c2c-4e08-9d85-c5ade81491a4\") " pod="openshift-insights/insights-runtime-extractor-trhgs"
Jul 28 19:59:45.489042 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:45.488922    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/be39a539-0c2c-4e08-9d85-c5ade81491a4-crio-socket\") pod \"insights-runtime-extractor-trhgs\" (UID: \"be39a539-0c2c-4e08-9d85-c5ade81491a4\") " pod="openshift-insights/insights-runtime-extractor-trhgs"
Jul 28 19:59:45.489042 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:45.488930    2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 19:59:45.489042 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:45.489025    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/be39a539-0c2c-4e08-9d85-c5ade81491a4-kube-rbac-proxy-cm podName:be39a539-0c2c-4e08-9d85-c5ade81491a4 nodeName:}" failed. No retries permitted until 2026-07-28 19:59:45.989002902 +0000 UTC m=+48.772014328 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/be39a539-0c2c-4e08-9d85-c5ade81491a4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-trhgs" (UID: "be39a539-0c2c-4e08-9d85-c5ade81491a4") : configmap "kube-rbac-proxy" not found
Jul 28 19:59:45.489245 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:45.489042    2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 19:59:45.489245 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:45.488957    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/be39a539-0c2c-4e08-9d85-c5ade81491a4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-trhgs\" (UID: \"be39a539-0c2c-4e08-9d85-c5ade81491a4\") " pod="openshift-insights/insights-runtime-extractor-trhgs"
Jul 28 19:59:45.489245 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:45.489113    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be39a539-0c2c-4e08-9d85-c5ade81491a4-insights-runtime-extractor-tls podName:be39a539-0c2c-4e08-9d85-c5ade81491a4 nodeName:}" failed. No retries permitted until 2026-07-28 19:59:45.98909611 +0000 UTC m=+48.772107545 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/be39a539-0c2c-4e08-9d85-c5ade81491a4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-trhgs" (UID: "be39a539-0c2c-4e08-9d85-c5ade81491a4") : secret "insights-runtime-extractor-tls" not found
Jul 28 19:59:45.489245 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:45.489139    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/be39a539-0c2c-4e08-9d85-c5ade81491a4-data-volume\") pod \"insights-runtime-extractor-trhgs\" (UID: \"be39a539-0c2c-4e08-9d85-c5ade81491a4\") " pod="openshift-insights/insights-runtime-extractor-trhgs"
Jul 28 19:59:45.499640 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:45.499609    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qm8n\" (UniqueName: \"kubernetes.io/projected/be39a539-0c2c-4e08-9d85-c5ade81491a4-kube-api-access-7qm8n\") pod \"insights-runtime-extractor-trhgs\" (UID: \"be39a539-0c2c-4e08-9d85-c5ade81491a4\") " pod="openshift-insights/insights-runtime-extractor-trhgs"
Jul 28 19:59:45.993284 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:45.993246    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/be39a539-0c2c-4e08-9d85-c5ade81491a4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-trhgs\" (UID: \"be39a539-0c2c-4e08-9d85-c5ade81491a4\") " pod="openshift-insights/insights-runtime-extractor-trhgs"
Jul 28 19:59:45.993491 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:45.993355    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/be39a539-0c2c-4e08-9d85-c5ade81491a4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-trhgs\" (UID: \"be39a539-0c2c-4e08-9d85-c5ade81491a4\") " pod="openshift-insights/insights-runtime-extractor-trhgs"
Jul 28 19:59:45.993491 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:45.993398    2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 19:59:45.993491 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:45.993476    2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 19:59:45.993624 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:45.993520    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/be39a539-0c2c-4e08-9d85-c5ade81491a4-kube-rbac-proxy-cm podName:be39a539-0c2c-4e08-9d85-c5ade81491a4 nodeName:}" failed. No retries permitted until 2026-07-28 19:59:46.993458658 +0000 UTC m=+49.776470091 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/be39a539-0c2c-4e08-9d85-c5ade81491a4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-trhgs" (UID: "be39a539-0c2c-4e08-9d85-c5ade81491a4") : configmap "kube-rbac-proxy" not found
Jul 28 19:59:45.993624 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:45.993550    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be39a539-0c2c-4e08-9d85-c5ade81491a4-insights-runtime-extractor-tls podName:be39a539-0c2c-4e08-9d85-c5ade81491a4 nodeName:}" failed. No retries permitted until 2026-07-28 19:59:46.993538467 +0000 UTC m=+49.776549892 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/be39a539-0c2c-4e08-9d85-c5ade81491a4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-trhgs" (UID: "be39a539-0c2c-4e08-9d85-c5ade81491a4") : secret "insights-runtime-extractor-tls" not found
Jul 28 19:59:46.193956 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:46.193930    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-7vcr8_dde4c1a8-8229-4419-87b6-d8e39a0827c4/console-operator/1.log"
Jul 28 19:59:46.194366 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:46.194356    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-7vcr8_dde4c1a8-8229-4419-87b6-d8e39a0827c4/console-operator/0.log"
Jul 28 19:59:46.194409 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:46.194390    2554 generic.go:356] "Generic (PLEG): container finished" podID="dde4c1a8-8229-4419-87b6-d8e39a0827c4" containerID="d6d3e6d271e1c365e2645f8eb01c217e1c963f8b219401651e6fab3418c9c03e" exitCode=255
Jul 28 19:59:46.194472 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:46.194455    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-7vcr8" event={"ID":"dde4c1a8-8229-4419-87b6-d8e39a0827c4","Type":"ContainerDied","Data":"d6d3e6d271e1c365e2645f8eb01c217e1c963f8b219401651e6fab3418c9c03e"}
Jul 28 19:59:46.194519 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:46.194508    2554 scope.go:117] "RemoveContainer" containerID="cbc995e403bcc5f23ef04446957f647cb49e3dcd84afe82102f3ec57bb51ea05"
Jul 28 19:59:46.194761 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:46.194710    2554 scope.go:117] "RemoveContainer" containerID="d6d3e6d271e1c365e2645f8eb01c217e1c963f8b219401651e6fab3418c9c03e"
Jul 28 19:59:46.194948 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:46.194923    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-7vcr8_openshift-console-operator(dde4c1a8-8229-4419-87b6-d8e39a0827c4)\"" pod="openshift-console-operator/console-operator-584dcb6b54-7vcr8" podUID="dde4c1a8-8229-4419-87b6-d8e39a0827c4"
Jul 28 19:59:46.754343 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:46.754308    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-42pjq_c254f770-a33c-4a75-bd75-9bdc3e4e12ed/dns-node-resolver/0.log"
Jul 28 19:59:47.001428 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:47.001388    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/be39a539-0c2c-4e08-9d85-c5ade81491a4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-trhgs\" (UID: \"be39a539-0c2c-4e08-9d85-c5ade81491a4\") " pod="openshift-insights/insights-runtime-extractor-trhgs"
Jul 28 19:59:47.001597 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:47.001474    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/be39a539-0c2c-4e08-9d85-c5ade81491a4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-trhgs\" (UID: \"be39a539-0c2c-4e08-9d85-c5ade81491a4\") " pod="openshift-insights/insights-runtime-extractor-trhgs"
Jul 28 19:59:47.001597 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:47.001518    2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 19:59:47.001597 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:47.001581    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/be39a539-0c2c-4e08-9d85-c5ade81491a4-kube-rbac-proxy-cm podName:be39a539-0c2c-4e08-9d85-c5ade81491a4 nodeName:}" failed. No retries permitted until 2026-07-28 19:59:49.001566432 +0000 UTC m=+51.784577852 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/be39a539-0c2c-4e08-9d85-c5ade81491a4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-trhgs" (UID: "be39a539-0c2c-4e08-9d85-c5ade81491a4") : configmap "kube-rbac-proxy" not found
Jul 28 19:59:47.001702 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:47.001619    2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 19:59:47.001702 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:47.001671    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be39a539-0c2c-4e08-9d85-c5ade81491a4-insights-runtime-extractor-tls podName:be39a539-0c2c-4e08-9d85-c5ade81491a4 nodeName:}" failed. No retries permitted until 2026-07-28 19:59:49.001658902 +0000 UTC m=+51.784670323 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/be39a539-0c2c-4e08-9d85-c5ade81491a4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-trhgs" (UID: "be39a539-0c2c-4e08-9d85-c5ade81491a4") : secret "insights-runtime-extractor-tls" not found
Jul 28 19:59:47.199030 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:47.199002    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-7vcr8_dde4c1a8-8229-4419-87b6-d8e39a0827c4/console-operator/1.log"
Jul 28 19:59:47.199433 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:47.199389    2554 scope.go:117] "RemoveContainer" containerID="d6d3e6d271e1c365e2645f8eb01c217e1c963f8b219401651e6fab3418c9c03e"
Jul 28 19:59:47.199570 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:47.199550    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-7vcr8_openshift-console-operator(dde4c1a8-8229-4419-87b6-d8e39a0827c4)\"" pod="openshift-console-operator/console-operator-584dcb6b54-7vcr8" podUID="dde4c1a8-8229-4419-87b6-d8e39a0827c4"
Jul 28 19:59:47.251524 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:47.251482    2554 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-7vcr8"
Jul 28 19:59:47.505391 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:47.505319    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2b7b89ba-b64b-454a-b032-34757974c00a-metrics-tls\") pod \"dns-default-qs4pv\" (UID: \"2b7b89ba-b64b-454a-b032-34757974c00a\") " pod="openshift-dns/dns-default-qs4pv"
Jul 28 19:59:47.505518 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:47.505432    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0dda64d8-2cc8-4164-afc1-303f060864a8-registry-tls\") pod \"image-registry-7bcc689c88-gmt7f\" (UID: \"0dda64d8-2cc8-4164-afc1-303f060864a8\") " pod="openshift-image-registry/image-registry-7bcc689c88-gmt7f"
Jul 28 19:59:47.505518 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:47.505467    2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 19:59:47.505591 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:47.505531    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b7b89ba-b64b-454a-b032-34757974c00a-metrics-tls podName:2b7b89ba-b64b-454a-b032-34757974c00a nodeName:}" failed. No retries permitted until 2026-07-28 20:00:03.505512469 +0000 UTC m=+66.288523906 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2b7b89ba-b64b-454a-b032-34757974c00a-metrics-tls") pod "dns-default-qs4pv" (UID: "2b7b89ba-b64b-454a-b032-34757974c00a") : secret "dns-default-metrics-tls" not found
Jul 28 19:59:47.505591 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:47.505562    2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 19:59:47.505591 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:47.505578    2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7bcc689c88-gmt7f: secret "image-registry-tls" not found
Jul 28 19:59:47.505699 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:47.505619    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0dda64d8-2cc8-4164-afc1-303f060864a8-registry-tls podName:0dda64d8-2cc8-4164-afc1-303f060864a8 nodeName:}" failed. No retries permitted until 2026-07-28 20:00:03.505608484 +0000 UTC m=+66.288619904 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0dda64d8-2cc8-4164-afc1-303f060864a8-registry-tls") pod "image-registry-7bcc689c88-gmt7f" (UID: "0dda64d8-2cc8-4164-afc1-303f060864a8") : secret "image-registry-tls" not found
Jul 28 19:59:48.154822 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:48.154792    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qrztd_38cf5bd2-15db-49bc-b41d-b0df3ea06cb1/node-ca/0.log"
Jul 28 19:59:48.202806 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:48.202776    2554 scope.go:117] "RemoveContainer" containerID="d6d3e6d271e1c365e2645f8eb01c217e1c963f8b219401651e6fab3418c9c03e"
Jul 28 19:59:48.203163 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:48.202939    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-7vcr8_openshift-console-operator(dde4c1a8-8229-4419-87b6-d8e39a0827c4)\"" pod="openshift-console-operator/console-operator-584dcb6b54-7vcr8" podUID="dde4c1a8-8229-4419-87b6-d8e39a0827c4"
Jul 28 19:59:48.754028 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:48.753996    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xg646_e69543ef-79c9-4a8f-bab6-8e1af517c029/serve-healthcheck-canary/0.log"
Jul 28 19:59:49.018826 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:49.018742    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/be39a539-0c2c-4e08-9d85-c5ade81491a4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-trhgs\" (UID: \"be39a539-0c2c-4e08-9d85-c5ade81491a4\") " pod="openshift-insights/insights-runtime-extractor-trhgs"
Jul 28 19:59:49.018959 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:49.018839    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/be39a539-0c2c-4e08-9d85-c5ade81491a4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-trhgs\" (UID: \"be39a539-0c2c-4e08-9d85-c5ade81491a4\") " pod="openshift-insights/insights-runtime-extractor-trhgs"
Jul 28 19:59:49.019008 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:49.018988    2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 19:59:49.019063 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:49.019053    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be39a539-0c2c-4e08-9d85-c5ade81491a4-insights-runtime-extractor-tls podName:be39a539-0c2c-4e08-9d85-c5ade81491a4 nodeName:}" failed. No retries permitted until 2026-07-28 19:59:53.0190386 +0000 UTC m=+55.802050020 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/be39a539-0c2c-4e08-9d85-c5ade81491a4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-trhgs" (UID: "be39a539-0c2c-4e08-9d85-c5ade81491a4") : secret "insights-runtime-extractor-tls" not found
Jul 28 19:59:49.019289 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:49.019272    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/be39a539-0c2c-4e08-9d85-c5ade81491a4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-trhgs\" (UID: \"be39a539-0c2c-4e08-9d85-c5ade81491a4\") " pod="openshift-insights/insights-runtime-extractor-trhgs"
Jul 28 19:59:52.859085 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:52.859032    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb24870f-3c94-4836-9e8b-8d6f757c84de-metrics-certs\") pod \"router-default-7c5b95cbd9-xk69c\" (UID: \"bb24870f-3c94-4836-9e8b-8d6f757c84de\") " pod="openshift-ingress/router-default-7c5b95cbd9-xk69c"
Jul 28 19:59:52.859604 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:52.859124    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb24870f-3c94-4836-9e8b-8d6f757c84de-service-ca-bundle\") pod \"router-default-7c5b95cbd9-xk69c\" (UID: \"bb24870f-3c94-4836-9e8b-8d6f757c84de\") " pod="openshift-ingress/router-default-7c5b95cbd9-xk69c"
Jul 28 19:59:52.859604 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:52.859198    2554 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 19:59:52.859604 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:52.859274    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb24870f-3c94-4836-9e8b-8d6f757c84de-metrics-certs podName:bb24870f-3c94-4836-9e8b-8d6f757c84de nodeName:}" failed. No retries permitted until 2026-07-28 20:00:08.85925405 +0000 UTC m=+71.642265489 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bb24870f-3c94-4836-9e8b-8d6f757c84de-metrics-certs") pod "router-default-7c5b95cbd9-xk69c" (UID: "bb24870f-3c94-4836-9e8b-8d6f757c84de") : secret "router-metrics-certs-default" not found
Jul 28 19:59:52.859604 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:52.859289    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bb24870f-3c94-4836-9e8b-8d6f757c84de-service-ca-bundle podName:bb24870f-3c94-4836-9e8b-8d6f757c84de nodeName:}" failed. No retries permitted until 2026-07-28 20:00:08.859283171 +0000 UTC m=+71.642294591 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/bb24870f-3c94-4836-9e8b-8d6f757c84de-service-ca-bundle") pod "router-default-7c5b95cbd9-xk69c" (UID: "bb24870f-3c94-4836-9e8b-8d6f757c84de") : configmap references non-existent config key: service-ca.crt
Jul 28 19:59:52.960675 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:52.960635    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/689af28a-582d-4dfe-a7f9-37cad631f34a-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-p4m9x\" (UID: \"689af28a-582d-4dfe-a7f9-37cad631f34a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-p4m9x"
Jul 28 19:59:52.960876 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:52.960691    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/15900109-73a5-4356-b125-6b73b7743570-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-5bj5s\" (UID: \"15900109-73a5-4356-b125-6b73b7743570\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-5bj5s"
Jul 28 19:59:52.960876 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:52.960789    2554 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 19:59:52.960876 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:52.960857    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/689af28a-582d-4dfe-a7f9-37cad631f34a-samples-operator-tls podName:689af28a-582d-4dfe-a7f9-37cad631f34a nodeName:}" failed. No retries permitted until 2026-07-28 20:00:08.960839963 +0000 UTC m=+71.743851383 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/689af28a-582d-4dfe-a7f9-37cad631f34a-samples-operator-tls") pod "cluster-samples-operator-6748489656-p4m9x" (UID: "689af28a-582d-4dfe-a7f9-37cad631f34a") : secret "samples-operator-tls" not found
Jul 28 19:59:52.961001 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:52.960880    2554 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 19:59:52.961001 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:52.960953    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15900109-73a5-4356-b125-6b73b7743570-cluster-monitoring-operator-tls podName:15900109-73a5-4356-b125-6b73b7743570 nodeName:}" failed. No retries permitted until 2026-07-28 20:00:08.960938795 +0000 UTC m=+71.743950236 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/15900109-73a5-4356-b125-6b73b7743570-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-5bj5s" (UID: "15900109-73a5-4356-b125-6b73b7743570") : secret "cluster-monitoring-operator-tls" not found
Jul 28 19:59:53.061736 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:53.061697    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/be39a539-0c2c-4e08-9d85-c5ade81491a4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-trhgs\" (UID: \"be39a539-0c2c-4e08-9d85-c5ade81491a4\") " pod="openshift-insights/insights-runtime-extractor-trhgs"
Jul 28 19:59:53.061924 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:53.061860    2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 19:59:53.061993 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:53.061937    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be39a539-0c2c-4e08-9d85-c5ade81491a4-insights-runtime-extractor-tls podName:be39a539-0c2c-4e08-9d85-c5ade81491a4 nodeName:}" failed. No retries permitted until 2026-07-28 20:00:01.061916341 +0000 UTC m=+63.844927760 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/be39a539-0c2c-4e08-9d85-c5ade81491a4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-trhgs" (UID: "be39a539-0c2c-4e08-9d85-c5ade81491a4") : secret "insights-runtime-extractor-tls" not found
Jul 28 19:59:54.176851 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:54.176819    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-7vcr8"
Jul 28 19:59:54.177338 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:54.177320    2554 scope.go:117] "RemoveContainer" containerID="d6d3e6d271e1c365e2645f8eb01c217e1c963f8b219401651e6fab3418c9c03e"
Jul 28 19:59:54.177513 ip-10-0-139-54 kubenswrapper[2554]: E0728 19:59:54.177492    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-7vcr8_openshift-console-operator(dde4c1a8-8229-4419-87b6-d8e39a0827c4)\"" pod="openshift-console-operator/console-operator-584dcb6b54-7vcr8" podUID="dde4c1a8-8229-4419-87b6-d8e39a0827c4"
Jul 28 19:59:58.115196 ip-10-0-139-54 kubenswrapper[2554]: I0728 19:59:58.115165    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qxw76"
Jul 28 20:00:01.135703 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:01.135666    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/be39a539-0c2c-4e08-9d85-c5ade81491a4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-trhgs\" (UID: \"be39a539-0c2c-4e08-9d85-c5ade81491a4\") " pod="openshift-insights/insights-runtime-extractor-trhgs"
Jul 28 20:00:01.137905 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:01.137876    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/be39a539-0c2c-4e08-9d85-c5ade81491a4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-trhgs\" (UID: \"be39a539-0c2c-4e08-9d85-c5ade81491a4\") " pod="openshift-insights/insights-runtime-extractor-trhgs"
Jul 28 20:00:01.273257 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:01.273224    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-mrvdk\""
Jul 28 20:00:01.280507 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:01.280476    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-trhgs"
Jul 28 20:00:01.404830 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:01.404801    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-trhgs"]
Jul 28 20:00:01.407369 ip-10-0-139-54 kubenswrapper[2554]: W0728 20:00:01.407339    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe39a539_0c2c_4e08_9d85_c5ade81491a4.slice/crio-ac956d4b13b59c14980ea4480ed8e06cab626ef8399ca202ef10b98fef040547 WatchSource:0}: Error finding container ac956d4b13b59c14980ea4480ed8e06cab626ef8399ca202ef10b98fef040547: Status 404 returned error can't find the container with id ac956d4b13b59c14980ea4480ed8e06cab626ef8399ca202ef10b98fef040547
Jul 28 20:00:02.244879 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:02.244843    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-trhgs" event={"ID":"be39a539-0c2c-4e08-9d85-c5ade81491a4","Type":"ContainerStarted","Data":"ac956d4b13b59c14980ea4480ed8e06cab626ef8399ca202ef10b98fef040547"}
Jul 28 20:00:02.448503 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:02.448464    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9cc5727b-a071-4793-a0b1-b92f6287080a-original-pull-secret\") pod \"global-pull-secret-syncer-rv7fr\" (UID: \"9cc5727b-a071-4793-a0b1-b92f6287080a\") " pod="kube-system/global-pull-secret-syncer-rv7fr"
Jul 28 20:00:02.448684 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:02.448573    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcjrs\" (UniqueName: \"kubernetes.io/projected/2f157e54-c70e-46b1-a41b-d60279cd7189-kube-api-access-qcjrs\") pod \"network-check-target-v2g2z\" (UID: \"2f157e54-c70e-46b1-a41b-d60279cd7189\") " pod="openshift-network-diagnostics/network-check-target-v2g2z"
Jul 28 20:00:02.451132 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:02.451107    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcjrs\" (UniqueName: \"kubernetes.io/projected/2f157e54-c70e-46b1-a41b-d60279cd7189-kube-api-access-qcjrs\") pod \"network-check-target-v2g2z\" (UID: \"2f157e54-c70e-46b1-a41b-d60279cd7189\") " pod="openshift-network-diagnostics/network-check-target-v2g2z"
Jul 28 20:00:02.451369 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:02.451351    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 20:00:02.461850 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:02.461827    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9cc5727b-a071-4793-a0b1-b92f6287080a-original-pull-secret\") pod \"global-pull-secret-syncer-rv7fr\" (UID: \"9cc5727b-a071-4793-a0b1-b92f6287080a\") " pod="kube-system/global-pull-secret-syncer-rv7fr"
Jul 28 20:00:02.550167 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:02.550130    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/752faedf-0642-47aa-88bb-72f0ae961ebb-metrics-certs\") pod \"network-metrics-daemon-xmd9l\" (UID: \"752faedf-0642-47aa-88bb-72f0ae961ebb\") " pod="openshift-multus/network-metrics-daemon-xmd9l"
Jul 28 20:00:02.553046 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:02.553017    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 20:00:02.563745 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:02.563715    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/752faedf-0642-47aa-88bb-72f0ae961ebb-metrics-certs\") pod \"network-metrics-daemon-xmd9l\" (UID: \"752faedf-0642-47aa-88bb-72f0ae961ebb\") " pod="openshift-multus/network-metrics-daemon-xmd9l"
Jul 28 20:00:02.689511 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:02.689482    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2kg9b\""
Jul 28 20:00:02.693939 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:02.693916    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rv7fr"
Jul 28 20:00:02.697048 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:02.697028    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v2g2z"
Jul 28 20:00:02.701815 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:02.701791    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gjwl7\""
Jul 28 20:00:02.709366 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:02.709343    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xmd9l"
Jul 28 20:00:02.865321 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:02.865242    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rv7fr"]
Jul 28 20:00:02.872778 ip-10-0-139-54 kubenswrapper[2554]: W0728 20:00:02.872743    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cc5727b_a071_4793_a0b1_b92f6287080a.slice/crio-bb476b0927d4ed8fd9ae64c242843df414ce44b2bc2477f8a810fec3e56aa34e WatchSource:0}: Error finding container bb476b0927d4ed8fd9ae64c242843df414ce44b2bc2477f8a810fec3e56aa34e: Status 404 returned error can't find the container with id bb476b0927d4ed8fd9ae64c242843df414ce44b2bc2477f8a810fec3e56aa34e
Jul 28 20:00:02.898436 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:02.898144    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-v2g2z"]
Jul 28 20:00:02.901141 ip-10-0-139-54 kubenswrapper[2554]: W0728 20:00:02.901102    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f157e54_c70e_46b1_a41b_d60279cd7189.slice/crio-f64fafb05d5ef2b81a8b26082ce797e2d144670c6558c8cf7b9167e0b5a4fc7f WatchSource:0}: Error finding container f64fafb05d5ef2b81a8b26082ce797e2d144670c6558c8cf7b9167e0b5a4fc7f: Status 404 returned error can't find the container with id f64fafb05d5ef2b81a8b26082ce797e2d144670c6558c8cf7b9167e0b5a4fc7f
Jul 28 20:00:02.932883 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:02.932784    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xmd9l"]
Jul 28 20:00:02.936101 ip-10-0-139-54 kubenswrapper[2554]: W0728 20:00:02.936055    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod752faedf_0642_47aa_88bb_72f0ae961ebb.slice/crio-a37437bd7719c3f292f0ef498090725892eba04e169be76405091d074b357568 WatchSource:0}: Error finding container a37437bd7719c3f292f0ef498090725892eba04e169be76405091d074b357568: Status 404 returned error can't find the container with id a37437bd7719c3f292f0ef498090725892eba04e169be76405091d074b357568
Jul 28 20:00:03.251873 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:03.251784    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-v2g2z" event={"ID":"2f157e54-c70e-46b1-a41b-d60279cd7189","Type":"ContainerStarted","Data":"5b752273a94a35e1c2555caf5fe767bb4d9ec573039d06f9bbc973210594c5f7"}
Jul 28 20:00:03.251873 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:03.251850    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-v2g2z" event={"ID":"2f157e54-c70e-46b1-a41b-d60279cd7189","Type":"ContainerStarted","Data":"f64fafb05d5ef2b81a8b26082ce797e2d144670c6558c8cf7b9167e0b5a4fc7f"}
Jul 28 20:00:03.252414 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:03.251879    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-v2g2z"
Jul 28 20:00:03.253420 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:03.253389    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xmd9l" event={"ID":"752faedf-0642-47aa-88bb-72f0ae961ebb","Type":"ContainerStarted","Data":"a37437bd7719c3f292f0ef498090725892eba04e169be76405091d074b357568"}
Jul 28 20:00:03.254684 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:03.254653    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rv7fr" event={"ID":"9cc5727b-a071-4793-a0b1-b92f6287080a","Type":"ContainerStarted","Data":"bb476b0927d4ed8fd9ae64c242843df414ce44b2bc2477f8a810fec3e56aa34e"}
Jul 28 20:00:03.269548 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:03.269456    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-v2g2z" podStartSLOduration=66.269440555 podStartE2EDuration="1m6.269440555s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:58:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:00:03.267950299 +0000 UTC m=+66.050961769" watchObservedRunningTime="2026-07-28 20:00:03.269440555 +0000 UTC m=+66.052451997"
Jul 28 20:00:03.557459 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:03.557425    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0dda64d8-2cc8-4164-afc1-303f060864a8-registry-tls\") pod \"image-registry-7bcc689c88-gmt7f\" (UID: \"0dda64d8-2cc8-4164-afc1-303f060864a8\") " pod="openshift-image-registry/image-registry-7bcc689c88-gmt7f"
Jul 28 20:00:03.557639 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:03.557503    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2b7b89ba-b64b-454a-b032-34757974c00a-metrics-tls\") pod \"dns-default-qs4pv\" (UID: \"2b7b89ba-b64b-454a-b032-34757974c00a\") " pod="openshift-dns/dns-default-qs4pv"
Jul 28 20:00:03.560044 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:03.560015    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2b7b89ba-b64b-454a-b032-34757974c00a-metrics-tls\") pod \"dns-default-qs4pv\" (UID: \"2b7b89ba-b64b-454a-b032-34757974c00a\") " pod="openshift-dns/dns-default-qs4pv"
Jul 28 20:00:03.560261 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:03.560239    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0dda64d8-2cc8-4164-afc1-303f060864a8-registry-tls\") pod \"image-registry-7bcc689c88-gmt7f\" (UID: \"0dda64d8-2cc8-4164-afc1-303f060864a8\") " pod="openshift-image-registry/image-registry-7bcc689c88-gmt7f"
Jul 28 20:00:03.685866 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:03.685824    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6f6rv\""
Jul 28 20:00:03.693188 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:03.693157    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7bcc689c88-gmt7f"
Jul 28 20:00:03.702470 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:03.702444    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ph26b\""
Jul 28 20:00:03.709988 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:03.709960    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qs4pv"
Jul 28 20:00:03.881257 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:03.881185    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7bcc689c88-gmt7f"]
Jul 28 20:00:03.884343 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:03.884310    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qs4pv"]
Jul 28 20:00:03.886381 ip-10-0-139-54 kubenswrapper[2554]: W0728 20:00:03.886317    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dda64d8_2cc8_4164_afc1_303f060864a8.slice/crio-96c1ae50457d724a8087dec30a5c1b8d2928e6e3c6ec5804ddadcec21aa4914f WatchSource:0}: Error finding container 96c1ae50457d724a8087dec30a5c1b8d2928e6e3c6ec5804ddadcec21aa4914f: Status 404 returned error can't find the container with id 96c1ae50457d724a8087dec30a5c1b8d2928e6e3c6ec5804ddadcec21aa4914f
Jul 28 20:00:03.889402 ip-10-0-139-54 kubenswrapper[2554]: W0728 20:00:03.889377    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b7b89ba_b64b_454a_b032_34757974c00a.slice/crio-c7921e9f627ca75497cde3b958526d3851c3af4d8113b9c86cf6a1bafe69289f WatchSource:0}: Error finding container c7921e9f627ca75497cde3b958526d3851c3af4d8113b9c86cf6a1bafe69289f: Status 404 returned error can't find the container with id c7921e9f627ca75497cde3b958526d3851c3af4d8113b9c86cf6a1bafe69289f
Jul 28 20:00:04.261760 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:04.261565    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7bcc689c88-gmt7f" event={"ID":"0dda64d8-2cc8-4164-afc1-303f060864a8","Type":"ContainerStarted","Data":"689dd5e43697577273194f0e783bec3663c45a09d135cc2409c0279f46ec676f"}
Jul 28 20:00:04.261760 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:04.261625    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7bcc689c88-gmt7f" event={"ID":"0dda64d8-2cc8-4164-afc1-303f060864a8","Type":"ContainerStarted","Data":"96c1ae50457d724a8087dec30a5c1b8d2928e6e3c6ec5804ddadcec21aa4914f"}
Jul 28 20:00:04.261760 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:04.261666    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7bcc689c88-gmt7f"
Jul 28 20:00:04.263830 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:04.263801    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qs4pv" event={"ID":"2b7b89ba-b64b-454a-b032-34757974c00a","Type":"ContainerStarted","Data":"c7921e9f627ca75497cde3b958526d3851c3af4d8113b9c86cf6a1bafe69289f"}
Jul 28 20:00:04.286449 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:04.286377    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7bcc689c88-gmt7f" podStartSLOduration=66.286361213 podStartE2EDuration="1m6.286361213s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:58:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:00:04.284186242 +0000 UTC m=+67.067197716" watchObservedRunningTime="2026-07-28 20:00:04.286361213 +0000 UTC m=+67.069372655"
Jul 28 20:00:04.772273 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:04.772236    2554 scope.go:117] "RemoveContainer" containerID="d6d3e6d271e1c365e2645f8eb01c217e1c963f8b219401651e6fab3418c9c03e"
Jul 28 20:00:05.269915 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:05.269870    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xmd9l" event={"ID":"752faedf-0642-47aa-88bb-72f0ae961ebb","Type":"ContainerStarted","Data":"8d46996b5cd0775f34ea00c90e8d9f31fa5d7382313ec74105809bc44099f53f"}
Jul 28 20:00:05.270399 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:05.269943    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xmd9l" event={"ID":"752faedf-0642-47aa-88bb-72f0ae961ebb","Type":"ContainerStarted","Data":"cf518df1a56a94ec30d659a2fb878b4575bcd1034f8d1da7a20d036da9f76007"}
Jul 28 20:00:05.287515 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:05.287435    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-xmd9l" podStartSLOduration=65.754779419 podStartE2EDuration="1m7.287413661s" totalImagesPullingTime="1.532634242s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:58:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:00:05.286536414 +0000 UTC m=+68.069547861" watchObservedRunningTime="2026-07-28 20:00:05.287413661 +0000 UTC m=+68.070425104"
Jul 28 20:00:08.909488 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:08.909396    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb24870f-3c94-4836-9e8b-8d6f757c84de-metrics-certs\") pod \"router-default-7c5b95cbd9-xk69c\" (UID: \"bb24870f-3c94-4836-9e8b-8d6f757c84de\") " pod="openshift-ingress/router-default-7c5b95cbd9-xk69c"
Jul 28 20:00:08.909488 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:08.909459    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb24870f-3c94-4836-9e8b-8d6f757c84de-service-ca-bundle\") pod \"router-default-7c5b95cbd9-xk69c\" (UID: \"bb24870f-3c94-4836-9e8b-8d6f757c84de\") " pod="openshift-ingress/router-default-7c5b95cbd9-xk69c"
Jul 28 20:00:08.910263 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:08.910237    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb24870f-3c94-4836-9e8b-8d6f757c84de-service-ca-bundle\") pod \"router-default-7c5b95cbd9-xk69c\" (UID: \"bb24870f-3c94-4836-9e8b-8d6f757c84de\") " pod="openshift-ingress/router-default-7c5b95cbd9-xk69c"
Jul 28 20:00:08.911744 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:08.911719    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb24870f-3c94-4836-9e8b-8d6f757c84de-metrics-certs\") pod \"router-default-7c5b95cbd9-xk69c\" (UID: \"bb24870f-3c94-4836-9e8b-8d6f757c84de\") " pod="openshift-ingress/router-default-7c5b95cbd9-xk69c"
Jul 28 20:00:09.010305 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:09.010263    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/689af28a-582d-4dfe-a7f9-37cad631f34a-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-p4m9x\" (UID: \"689af28a-582d-4dfe-a7f9-37cad631f34a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-p4m9x"
Jul 28 20:00:09.010492 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:09.010319    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/15900109-73a5-4356-b125-6b73b7743570-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-5bj5s\" (UID: \"15900109-73a5-4356-b125-6b73b7743570\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-5bj5s"
Jul 28 20:00:09.012856 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:09.012827    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/15900109-73a5-4356-b125-6b73b7743570-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-5bj5s\" (UID: \"15900109-73a5-4356-b125-6b73b7743570\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-5bj5s"
Jul 28 20:00:09.013002 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:09.012977    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/689af28a-582d-4dfe-a7f9-37cad631f34a-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-p4m9x\" (UID: \"689af28a-582d-4dfe-a7f9-37cad631f34a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-p4m9x"
Jul 28 20:00:09.158281 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:09.158252    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-v4hr8\""
Jul 28 20:00:09.165739 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:09.165660    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7c5b95cbd9-xk69c"
Jul 28 20:00:09.232284 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:09.232256    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-rjj5t\""
Jul 28 20:00:09.236833 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:09.236812    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-fvtk6\""
Jul 28 20:00:09.240015 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:09.239990    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-5bj5s"
Jul 28 20:00:09.245132 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:09.245112    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-p4m9x"
Jul 28 20:00:09.956423 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:09.955471    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7c5b95cbd9-xk69c"]
Jul 28 20:00:09.982002 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:09.981935    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-5bj5s"]
Jul 28 20:00:09.984781 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:09.983959    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-p4m9x"]
Jul 28 20:00:09.986997 ip-10-0-139-54 kubenswrapper[2554]: W0728 20:00:09.986966    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15900109_73a5_4356_b125_6b73b7743570.slice/crio-8424c2c06d2a1d4512c292bd980efb534cca82d2c9aa870d352fd2777c2c266d WatchSource:0}: Error finding container 8424c2c06d2a1d4512c292bd980efb534cca82d2c9aa870d352fd2777c2c266d: Status 404 returned error can't find the container with id 8424c2c06d2a1d4512c292bd980efb534cca82d2c9aa870d352fd2777c2c266d
Jul 28 20:00:10.290915 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:10.290880    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7c5b95cbd9-xk69c" event={"ID":"bb24870f-3c94-4836-9e8b-8d6f757c84de","Type":"ContainerStarted","Data":"1827537e9e4ef3ee088a3c50563c11099916065244c1d452514c4f741568a4bc"}
Jul 28 20:00:10.291115 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:10.290931    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7c5b95cbd9-xk69c" event={"ID":"bb24870f-3c94-4836-9e8b-8d6f757c84de","Type":"ContainerStarted","Data":"cdad709ff7fd2571fe542ce59a115e8da76b0f460edc8eb76ec5effb2fbae3ab"}
Jul 28 20:00:10.292914 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:10.292884    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-trhgs" event={"ID":"be39a539-0c2c-4e08-9d85-c5ade81491a4","Type":"ContainerStarted","Data":"91664138e920f3453c62384c793abb339de4d1eb8851a9daeb2aa43835092db6"}
Jul 28 20:00:10.294627 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:10.294584    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rv7fr" event={"ID":"9cc5727b-a071-4793-a0b1-b92f6287080a","Type":"ContainerStarted","Data":"7599498066c4fa060a54a7352413dcadfb5b0f76dcf1f3ad0d2c0c1f143d34eb"}
Jul 28 20:00:10.296127 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:10.296093    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-5bj5s" event={"ID":"15900109-73a5-4356-b125-6b73b7743570","Type":"ContainerStarted","Data":"8424c2c06d2a1d4512c292bd980efb534cca82d2c9aa870d352fd2777c2c266d"}
Jul 28 20:00:10.298035 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:10.298006    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qs4pv" event={"ID":"2b7b89ba-b64b-454a-b032-34757974c00a","Type":"ContainerStarted","Data":"92a336a205144095250589da0611026b46b0f6531caa6c1f63ab0ee03403341b"}
Jul 28 20:00:10.298159 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:10.298088    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-qs4pv"
Jul 28 20:00:10.298159 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:10.298117    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qs4pv" event={"ID":"2b7b89ba-b64b-454a-b032-34757974c00a","Type":"ContainerStarted","Data":"1029608cae17f818bea5a6acb2a141742e1dbf74b9d974e604cd31d9002636f1"}
Jul 28 20:00:10.299641 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:10.299616    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-p4m9x" event={"ID":"689af28a-582d-4dfe-a7f9-37cad631f34a","Type":"ContainerStarted","Data":"182500e8ed8a3c7edbd6a65060d52c79dfde5fa0f512a67ccd7c591ff62cbc45"}
Jul 28 20:00:10.301727 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:10.301703    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-7vcr8_dde4c1a8-8229-4419-87b6-d8e39a0827c4/console-operator/1.log"
Jul 28 20:00:10.301804 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:10.301763    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-7vcr8" event={"ID":"dde4c1a8-8229-4419-87b6-d8e39a0827c4","Type":"ContainerStarted","Data":"8cc72ac7b9fda57cd272c773de470c9ed2c3a8b1da752562f796c4e536fdfcef"}
Jul 28 20:00:10.302046 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:10.302017    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-7vcr8"
Jul 28 20:00:10.307136 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:10.307119    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-7vcr8"
Jul 28 20:00:10.310971 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:10.310840    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7c5b95cbd9-xk69c" podStartSLOduration=34.310827214 podStartE2EDuration="34.310827214s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:59:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:00:10.310100155 +0000 UTC m=+73.093111594" watchObservedRunningTime="2026-07-28 20:00:10.310827214 +0000 UTC m=+73.093838655"
Jul 28 20:00:10.329210 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:10.329160    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-rv7fr" podStartSLOduration=65.465349471 podStartE2EDuration="1m12.329139421s" totalImagesPullingTime="6.86378995s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:58:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:00:10.328867652 +0000 UTC m=+73.111879114" watchObservedRunningTime="2026-07-28 20:00:10.329139421 +0000 UTC m=+73.112150866"
Jul 28 20:00:10.347472 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:10.347422    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-qs4pv" podStartSLOduration=33.528333577 podStartE2EDuration="39.34740458s" totalImagesPullingTime="5.819071003s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:59:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:00:10.346948889 +0000 UTC m=+73.129960352" watchObservedRunningTime="2026-07-28 20:00:10.34740458 +0000 UTC m=+73.130416022"
Jul 28 20:00:10.401290 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:10.400095    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-zdppx"]
Jul 28 20:00:10.402733 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:10.402650    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-zdppx"
Jul 28 20:00:10.406096 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:10.405965    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-gsstz\""
Jul 28 20:00:10.407313 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:10.406253    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 28 20:00:10.407313 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:10.406518    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 28 20:00:10.416421 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:10.416334    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-zdppx"]
Jul 28 20:00:10.521876 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:10.521836    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22htd\" (UniqueName: \"kubernetes.io/projected/780efb21-e22d-46d1-b7c2-4ab44c7f4600-kube-api-access-22htd\") pod \"downloads-7c97dc4555-zdppx\" (UID: \"780efb21-e22d-46d1-b7c2-4ab44c7f4600\") " pod="openshift-console/downloads-7c97dc4555-zdppx"
Jul 28 20:00:10.622726 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:10.622638    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22htd\" (UniqueName: \"kubernetes.io/projected/780efb21-e22d-46d1-b7c2-4ab44c7f4600-kube-api-access-22htd\") pod \"downloads-7c97dc4555-zdppx\" (UID: \"780efb21-e22d-46d1-b7c2-4ab44c7f4600\") " pod="openshift-console/downloads-7c97dc4555-zdppx"
Jul 28 20:00:10.632356 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:10.632302    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22htd\" (UniqueName: \"kubernetes.io/projected/780efb21-e22d-46d1-b7c2-4ab44c7f4600-kube-api-access-22htd\") pod \"downloads-7c97dc4555-zdppx\" (UID: \"780efb21-e22d-46d1-b7c2-4ab44c7f4600\") " pod="openshift-console/downloads-7c97dc4555-zdppx"
Jul 28 20:00:10.715601 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:10.715379    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-zdppx"
Jul 28 20:00:10.875335 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:10.875220    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-zdppx"]
Jul 28 20:00:10.877997 ip-10-0-139-54 kubenswrapper[2554]: W0728 20:00:10.877964    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod780efb21_e22d_46d1_b7c2_4ab44c7f4600.slice/crio-3a9011790a0f56c4a0fce61678dc30bc476046ce0d489ee07cb54c35486436c4 WatchSource:0}: Error finding container 3a9011790a0f56c4a0fce61678dc30bc476046ce0d489ee07cb54c35486436c4: Status 404 returned error can't find the container with id 3a9011790a0f56c4a0fce61678dc30bc476046ce0d489ee07cb54c35486436c4
Jul 28 20:00:11.166772 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:11.166690    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7c5b95cbd9-xk69c"
Jul 28 20:00:11.169793 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:11.169587    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7c5b95cbd9-xk69c"
Jul 28 20:00:11.309939 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:11.308935    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-zdppx" event={"ID":"780efb21-e22d-46d1-b7c2-4ab44c7f4600","Type":"ContainerStarted","Data":"3a9011790a0f56c4a0fce61678dc30bc476046ce0d489ee07cb54c35486436c4"}
Jul 28 20:00:11.310786 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:11.310124    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7c5b95cbd9-xk69c"
Jul 28 20:00:11.311943 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:11.311753    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7c5b95cbd9-xk69c"
Jul 28 20:00:15.329744 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:15.329660    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-trhgs" event={"ID":"be39a539-0c2c-4e08-9d85-c5ade81491a4","Type":"ContainerStarted","Data":"637320c0e589fcb7e3146187cb71e78c8c927e2a1ededdee2230069ee7926d9b"}
Jul 28 20:00:15.331801 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:15.331766    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-5bj5s" event={"ID":"15900109-73a5-4356-b125-6b73b7743570","Type":"ContainerStarted","Data":"677130f890690f14800825f82868efc457ea9c7477a417a51472c723bb0ee023"}
Jul 28 20:00:15.333729 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:15.333674    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-p4m9x" event={"ID":"689af28a-582d-4dfe-a7f9-37cad631f34a","Type":"ContainerStarted","Data":"369d80435d42838241d22dc9cc15e26d544eae9c43a2383de92b92b4f5212c10"}
Jul 28 20:00:15.333729 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:15.333725    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-p4m9x" event={"ID":"689af28a-582d-4dfe-a7f9-37cad631f34a","Type":"ContainerStarted","Data":"f834d4cc6ce0ff3b82f85542175143ff7ca62126c06eeca72a5d5132e3ed0eac"}
Jul 28 20:00:15.366368 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:15.366197    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-5bj5s" podStartSLOduration=33.433299239 podStartE2EDuration="38.366175988s" totalImagesPullingTime="4.932876749s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:59:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:00:15.348968338 +0000 UTC m=+78.131979806" watchObservedRunningTime="2026-07-28 20:00:15.366175988 +0000 UTC m=+78.149187432"
Jul 28 20:00:15.366764 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:15.366717    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-p4m9x" podStartSLOduration=33.49957241 podStartE2EDuration="38.366704197s" totalImagesPullingTime="4.867131787s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:59:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:00:15.366019834 +0000 UTC m=+78.149031300" watchObservedRunningTime="2026-07-28 20:00:15.366704197 +0000 UTC m=+78.149715640"
Jul 28 20:00:20.312235 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:20.312201    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-qs4pv"
Jul 28 20:00:20.357598 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:20.357556    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-trhgs" event={"ID":"be39a539-0c2c-4e08-9d85-c5ade81491a4","Type":"ContainerStarted","Data":"b53a9e55f9945cd84b0cd4bc747bde99ec2b3f64c5c13fc4875abb8e33e1191f"}
Jul 28 20:00:20.375599 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:20.375538    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-trhgs" podStartSLOduration=17.138401571 podStartE2EDuration="35.375517944s" totalImagesPullingTime="18.237116373s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:59:45 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:00:20.37501267 +0000 UTC m=+83.158024141" watchObservedRunningTime="2026-07-28 20:00:20.375517944 +0000 UTC m=+83.158529388"
Jul 28 20:00:25.274890 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:25.274859    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7bcc689c88-gmt7f"
Jul 28 20:00:29.395944 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:29.395848    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-zdppx" event={"ID":"780efb21-e22d-46d1-b7c2-4ab44c7f4600","Type":"ContainerStarted","Data":"0706c41ddd02e36d4102fe76cf856223024d82dff1e290fc2e615a17fbe0def3"}
Jul 28 20:00:29.396431 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:29.396110    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-zdppx"
Jul 28 20:00:29.412145 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:29.412121    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-zdppx"
Jul 28 20:00:29.418595 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:29.418534    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-zdppx" podStartSLOduration=1.2507087669999999 podStartE2EDuration="19.418516708s" totalImagesPullingTime="18.167807941s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:00:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:00:29.418415824 +0000 UTC m=+92.201427258" watchObservedRunningTime="2026-07-28 20:00:29.418516708 +0000 UTC m=+92.201528145"
Jul 28 20:00:29.944033 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:29.943995    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7bcc689c88-gmt7f"]
Jul 28 20:00:32.954022 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:32.953983    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-pv6xj"]
Jul 28 20:00:32.981050 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:32.981019    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-pv6xj"
Jul 28 20:00:32.984099 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:32.984055    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 28 20:00:32.984234 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:32.984101    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-q6dkm\""
Jul 28 20:00:32.984234 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:32.984176    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 28 20:00:32.984234 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:32.984055    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 28 20:00:32.984467 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:32.984443    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 28 20:00:33.014827 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:33.014790    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd-node-exporter-wtmp\") pod \"node-exporter-pv6xj\" (UID: \"cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd\") " pod="openshift-monitoring/node-exporter-pv6xj"
Jul 28 20:00:33.015005 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:33.014861    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd-metrics-client-ca\") pod \"node-exporter-pv6xj\" (UID: \"cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd\") " pod="openshift-monitoring/node-exporter-pv6xj"
Jul 28 20:00:33.015005 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:33.014893    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd-node-exporter-tls\") pod \"node-exporter-pv6xj\" (UID: \"cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd\") " pod="openshift-monitoring/node-exporter-pv6xj"
Jul 28 20:00:33.015005 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:33.014938    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd-root\") pod \"node-exporter-pv6xj\" (UID: \"cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd\") " pod="openshift-monitoring/node-exporter-pv6xj"
Jul 28 20:00:33.015194 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:33.015008    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd-node-exporter-textfile\") pod \"node-exporter-pv6xj\" (UID: \"cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd\") " pod="openshift-monitoring/node-exporter-pv6xj"
Jul 28 20:00:33.015194 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:33.015049    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pv6xj\" (UID: \"cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd\") " pod="openshift-monitoring/node-exporter-pv6xj"
Jul 28 20:00:33.015194 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:33.015109    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd-node-exporter-accelerators-collector-config\") pod \"node-exporter-pv6xj\" (UID: \"cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd\") " pod="openshift-monitoring/node-exporter-pv6xj"
Jul 28 20:00:33.015194 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:33.015144    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccmkk\" (UniqueName: \"kubernetes.io/projected/cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd-kube-api-access-ccmkk\") pod \"node-exporter-pv6xj\" (UID: \"cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd\") " pod="openshift-monitoring/node-exporter-pv6xj"
Jul 28 20:00:33.015194 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:33.015180    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd-sys\") pod \"node-exporter-pv6xj\" (UID: \"cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd\") " pod="openshift-monitoring/node-exporter-pv6xj"
Jul 28 20:00:33.115756 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:33.115710    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd-metrics-client-ca\") pod \"node-exporter-pv6xj\" (UID: \"cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd\") " pod="openshift-monitoring/node-exporter-pv6xj"
Jul 28 20:00:33.115936 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:33.115767    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd-node-exporter-tls\") pod \"node-exporter-pv6xj\" (UID: \"cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd\") " pod="openshift-monitoring/node-exporter-pv6xj"
Jul 28 20:00:33.115936 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:33.115801    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd-root\") pod \"node-exporter-pv6xj\" (UID: \"cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd\") " pod="openshift-monitoring/node-exporter-pv6xj"
Jul 28 20:00:33.115936 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:33.115832    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd-node-exporter-textfile\") pod \"node-exporter-pv6xj\" (UID: \"cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd\") " pod="openshift-monitoring/node-exporter-pv6xj"
Jul 28 20:00:33.115936 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:33.115855    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pv6xj\" (UID: \"cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd\") " pod="openshift-monitoring/node-exporter-pv6xj"
Jul 28 20:00:33.115936 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:33.115896    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd-node-exporter-accelerators-collector-config\") pod \"node-exporter-pv6xj\" (UID: \"cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd\") " pod="openshift-monitoring/node-exporter-pv6xj"
Jul 28 20:00:33.115936 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:33.115928    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccmkk\" (UniqueName: \"kubernetes.io/projected/cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd-kube-api-access-ccmkk\") pod \"node-exporter-pv6xj\" (UID: \"cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd\") " pod="openshift-monitoring/node-exporter-pv6xj"
Jul 28 20:00:33.116275 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:33.115961    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd-sys\") pod \"node-exporter-pv6xj\" (UID: \"cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd\") " pod="openshift-monitoring/node-exporter-pv6xj"
Jul 28 20:00:33.116275 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:33.116016    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd-node-exporter-wtmp\") pod \"node-exporter-pv6xj\" (UID: \"cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd\") " pod="openshift-monitoring/node-exporter-pv6xj"
Jul 28 20:00:33.116275 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:33.116247    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd-node-exporter-wtmp\") pod \"node-exporter-pv6xj\" (UID: \"cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd\") " pod="openshift-monitoring/node-exporter-pv6xj"
Jul 28 20:00:33.116452 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:33.116432    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd-metrics-client-ca\") pod \"node-exporter-pv6xj\" (UID: \"cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd\") " pod="openshift-monitoring/node-exporter-pv6xj"
Jul 28 20:00:33.116822 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:33.116618    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd-sys\") pod \"node-exporter-pv6xj\" (UID: \"cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd\") " pod="openshift-monitoring/node-exporter-pv6xj"
Jul 28 20:00:33.116822 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:33.116690    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd-root\") pod \"node-exporter-pv6xj\" (UID: \"cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd\") " pod="openshift-monitoring/node-exporter-pv6xj"
Jul 28 20:00:33.116822 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:33.116754    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd-node-exporter-accelerators-collector-config\") pod \"node-exporter-pv6xj\" (UID: \"cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd\") " pod="openshift-monitoring/node-exporter-pv6xj"
Jul 28 20:00:33.116996 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:33.116917    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd-node-exporter-textfile\") pod \"node-exporter-pv6xj\" (UID: \"cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd\") " pod="openshift-monitoring/node-exporter-pv6xj"
Jul 28 20:00:33.119317 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:33.119294    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd-node-exporter-tls\") pod \"node-exporter-pv6xj\" (UID: \"cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd\") " pod="openshift-monitoring/node-exporter-pv6xj"
Jul 28 20:00:33.119620 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:33.119601    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pv6xj\" (UID: \"cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd\") " pod="openshift-monitoring/node-exporter-pv6xj"
Jul 28 20:00:33.129802 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:33.129775    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccmkk\" (UniqueName: \"kubernetes.io/projected/cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd-kube-api-access-ccmkk\") pod \"node-exporter-pv6xj\" (UID: \"cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd\") " pod="openshift-monitoring/node-exporter-pv6xj"
Jul 28 20:00:33.292296 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:33.292259    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-pv6xj"
Jul 28 20:00:33.327881 ip-10-0-139-54 kubenswrapper[2554]: W0728 20:00:33.327843    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb6b59a5_2a48_4d15_8dd9_4c19242bc9cd.slice/crio-0c434ef443335eeeaee06ce295b664986176df2995530c2e15f00cb3727a0811 WatchSource:0}: Error finding container 0c434ef443335eeeaee06ce295b664986176df2995530c2e15f00cb3727a0811: Status 404 returned error can't find the container with id 0c434ef443335eeeaee06ce295b664986176df2995530c2e15f00cb3727a0811
Jul 28 20:00:33.414838 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:33.414796    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pv6xj" event={"ID":"cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd","Type":"ContainerStarted","Data":"0c434ef443335eeeaee06ce295b664986176df2995530c2e15f00cb3727a0811"}
Jul 28 20:00:34.268172 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:34.267339    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-v2g2z"
Jul 28 20:00:35.426191 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:35.426105    2554 generic.go:356] "Generic (PLEG): container finished" podID="cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd" containerID="fd189ee7a0cc93d59022a9b28fbae8e2e858809060e7ab5f5eed8f22a76611bb" exitCode=0
Jul 28 20:00:35.426623 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:35.426200    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pv6xj" event={"ID":"cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd","Type":"ContainerDied","Data":"fd189ee7a0cc93d59022a9b28fbae8e2e858809060e7ab5f5eed8f22a76611bb"}
Jul 28 20:00:36.433686 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:36.433646    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pv6xj" event={"ID":"cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd","Type":"ContainerStarted","Data":"2befed06e63e998079fb9df79eca09b183bf26d05c15a35aaea07a2df70b103e"}
Jul 28 20:00:36.434198 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:36.433711    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pv6xj" event={"ID":"cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd","Type":"ContainerStarted","Data":"f2a63a8036c44a5b4fbf7e1b82545a7d83c0e41f9c2cac6cc25a34df31887709"}
Jul 28 20:00:36.458212 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:36.458153    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-pv6xj" podStartSLOduration=2.581624406 podStartE2EDuration="4.458129464s" totalImagesPullingTime="1.807072345s" totalInitContainerRuntime="69.432713ms" isStatefulPod=true podCreationTimestamp="2026-07-28 20:00:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:00:36.455760146 +0000 UTC m=+99.238771617" watchObservedRunningTime="2026-07-28 20:00:36.458129464 +0000 UTC m=+99.241141006"
Jul 28 20:00:39.099939 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.099898    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 20:00:39.166765 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.166731    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 20:00:39.166931 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.166890    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.170062 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.170036    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 28 20:00:39.170187 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.170157    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-6trmh4jod19jl\""
Jul 28 20:00:39.170508 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.170493    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 28 20:00:39.170593 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.170493    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 28 20:00:39.170706 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.170693    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 28 20:00:39.170899 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.170880    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 28 20:00:39.171036 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.170927    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 28 20:00:39.171036 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.170938    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-p9hzm\""
Jul 28 20:00:39.171036 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.170929    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 28 20:00:39.171036 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.171014    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 28 20:00:39.172232 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.172205    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 28 20:00:39.172352 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.172269    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 28 20:00:39.172352 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.172269    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 28 20:00:39.172352 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.172316    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 28 20:00:39.172567 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.172551    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 28 20:00:39.174524 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.174504    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 28 20:00:39.177292 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.177271    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 28 20:00:39.275342 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.275305    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.275342 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.275340    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.275558 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.275361    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.275558 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.275394    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/07c3bf7b-8b01-422a-ac70-04aa061e488f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.275558 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.275455    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/07c3bf7b-8b01-422a-ac70-04aa061e488f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.275558 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.275505    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07c3bf7b-8b01-422a-ac70-04aa061e488f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.275558 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.275528    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-config\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.275558 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.275550    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.275760 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.275642    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-web-config\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.275760 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.275666    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07c3bf7b-8b01-422a-ac70-04aa061e488f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.275760 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.275694    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.275760 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.275715    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/07c3bf7b-8b01-422a-ac70-04aa061e488f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.275760 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.275734    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/07c3bf7b-8b01-422a-ac70-04aa061e488f-config-out\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.275760 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.275754    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/07c3bf7b-8b01-422a-ac70-04aa061e488f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.275994 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.275794    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/07c3bf7b-8b01-422a-ac70-04aa061e488f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.275994 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.275816    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.275994 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.275839    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/07c3bf7b-8b01-422a-ac70-04aa061e488f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.275994 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.275858    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07c3bf7b-8b01-422a-ac70-04aa061e488f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.275994 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.275875    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r62n8\" (UniqueName: \"kubernetes.io/projected/07c3bf7b-8b01-422a-ac70-04aa061e488f-kube-api-access-r62n8\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.275994 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.275893    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.377141 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.377046    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/07c3bf7b-8b01-422a-ac70-04aa061e488f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.377297 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.377181    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/07c3bf7b-8b01-422a-ac70-04aa061e488f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.377297 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.377199    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.377297 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.377222    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/07c3bf7b-8b01-422a-ac70-04aa061e488f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.377297 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.377238    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07c3bf7b-8b01-422a-ac70-04aa061e488f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.377297 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.377254    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r62n8\" (UniqueName: \"kubernetes.io/projected/07c3bf7b-8b01-422a-ac70-04aa061e488f-kube-api-access-r62n8\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.377297 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.377272    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.377584 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.377336    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.377584 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.377360    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.377584 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.377389    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.377584 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.377427    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/07c3bf7b-8b01-422a-ac70-04aa061e488f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.377584 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.377459    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/07c3bf7b-8b01-422a-ac70-04aa061e488f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.377584 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.377456    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/07c3bf7b-8b01-422a-ac70-04aa061e488f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.377584 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.377496    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07c3bf7b-8b01-422a-ac70-04aa061e488f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.377584 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.377529    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-config\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.377584 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.377560    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.378005 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.377608    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-web-config\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.378005 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.377627    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07c3bf7b-8b01-422a-ac70-04aa061e488f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.378005 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.377654    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.378005 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.377719    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/07c3bf7b-8b01-422a-ac70-04aa061e488f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.378005 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.377748    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/07c3bf7b-8b01-422a-ac70-04aa061e488f-config-out\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.378653 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.378625    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07c3bf7b-8b01-422a-ac70-04aa061e488f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.379597 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.379567    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07c3bf7b-8b01-422a-ac70-04aa061e488f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.380500 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.380182    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/07c3bf7b-8b01-422a-ac70-04aa061e488f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.381023 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.380767    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/07c3bf7b-8b01-422a-ac70-04aa061e488f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.381023 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.380807    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07c3bf7b-8b01-422a-ac70-04aa061e488f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.381023 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.380976    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/07c3bf7b-8b01-422a-ac70-04aa061e488f-config-out\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.381253 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.381108    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.381310 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.381248    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-web-config\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.382049 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.381658    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.382049 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.381665    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.382049 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.381786    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.382049 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.382005    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/07c3bf7b-8b01-422a-ac70-04aa061e488f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.382308 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.382290    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/07c3bf7b-8b01-422a-ac70-04aa061e488f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.382876 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.382848    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.382876 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.382864    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.383332 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.383312    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.384425 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.384402    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/07c3bf7b-8b01-422a-ac70-04aa061e488f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.384495 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.384433    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-config\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.385391 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.385374    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r62n8\" (UniqueName: \"kubernetes.io/projected/07c3bf7b-8b01-422a-ac70-04aa061e488f-kube-api-access-r62n8\") pod \"prometheus-k8s-0\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.477674 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.477626    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:39.640457 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:39.640378    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 20:00:39.643767 ip-10-0-139-54 kubenswrapper[2554]: W0728 20:00:39.643741    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07c3bf7b_8b01_422a_ac70_04aa061e488f.slice/crio-c64f499107984111fd92900a9da8c196b7c3175115ebfb089065aa23c213d7ff WatchSource:0}: Error finding container c64f499107984111fd92900a9da8c196b7c3175115ebfb089065aa23c213d7ff: Status 404 returned error can't find the container with id c64f499107984111fd92900a9da8c196b7c3175115ebfb089065aa23c213d7ff
Jul 28 20:00:40.450176 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:40.450128    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"07c3bf7b-8b01-422a-ac70-04aa061e488f","Type":"ContainerStarted","Data":"c64f499107984111fd92900a9da8c196b7c3175115ebfb089065aa23c213d7ff"}
Jul 28 20:00:41.458486 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:41.458367    2554 generic.go:356] "Generic (PLEG): container finished" podID="07c3bf7b-8b01-422a-ac70-04aa061e488f" containerID="a633dbf7a8f980361a74363cda62edec760ccd9ed470b4464fb1bdc423090fb9" exitCode=0
Jul 28 20:00:41.458973 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:41.458468    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"07c3bf7b-8b01-422a-ac70-04aa061e488f","Type":"ContainerDied","Data":"a633dbf7a8f980361a74363cda62edec760ccd9ed470b4464fb1bdc423090fb9"}
Jul 28 20:00:45.487416 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:45.487320    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"07c3bf7b-8b01-422a-ac70-04aa061e488f","Type":"ContainerStarted","Data":"d841b38735c03b304249adb9640c46d374a668692f3128b6d7540bcde592e4ea"}
Jul 28 20:00:46.494348 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:46.494304    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"07c3bf7b-8b01-422a-ac70-04aa061e488f","Type":"ContainerStarted","Data":"33ba837ca10f256b80049701ee6944b1a0612b2ec5b48d57941bdff4ce730763"}
Jul 28 20:00:48.504515 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:48.504417    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"07c3bf7b-8b01-422a-ac70-04aa061e488f","Type":"ContainerStarted","Data":"c8230091c31c7f2421b1227b5bcdfd8522650f665721c7763cb157163888f113"}
Jul 28 20:00:48.504515 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:48.504493    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"07c3bf7b-8b01-422a-ac70-04aa061e488f","Type":"ContainerStarted","Data":"b0b4ad53db8bb0123338df8dbb727d4bae7e17875d5551bccbce138c8c36fbf4"}
Jul 28 20:00:48.505000 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:48.504533    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"07c3bf7b-8b01-422a-ac70-04aa061e488f","Type":"ContainerStarted","Data":"80aa6654c5840b30f1b786b81956267cf43ee950fac48b8b48d8fb1dd20bb7f2"}
Jul 28 20:00:48.505000 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:48.504569    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"07c3bf7b-8b01-422a-ac70-04aa061e488f","Type":"ContainerStarted","Data":"b4031d15d133ac43134fafeb273f03afedd787f1cd321bc6ca36c19ba2d13d0c"}
Jul 28 20:00:48.532287 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:48.532234    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.091920383 podStartE2EDuration="9.53221552s" totalImagesPullingTime="7.41678684s" totalInitContainerRuntime="23.508297ms" isStatefulPod=false podCreationTimestamp="2026-07-28 20:00:39 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:00:48.530797424 +0000 UTC m=+111.313808886" watchObservedRunningTime="2026-07-28 20:00:48.53221552 +0000 UTC m=+111.315226963"
Jul 28 20:00:49.478037 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:49.477995    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:00:54.967983 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:54.967932    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7bcc689c88-gmt7f" podUID="0dda64d8-2cc8-4164-afc1-303f060864a8" containerName="registry" containerID="cri-o://689dd5e43697577273194f0e783bec3663c45a09d135cc2409c0279f46ec676f" gracePeriod=30
Jul 28 20:00:55.260093 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:55.260046    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7bcc689c88-gmt7f"
Jul 28 20:00:55.426986 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:55.426948    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0dda64d8-2cc8-4164-afc1-303f060864a8-ca-trust-extracted\") pod \"0dda64d8-2cc8-4164-afc1-303f060864a8\" (UID: \"0dda64d8-2cc8-4164-afc1-303f060864a8\") "
Jul 28 20:00:55.427217 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:55.427046    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0dda64d8-2cc8-4164-afc1-303f060864a8-trusted-ca\") pod \"0dda64d8-2cc8-4164-afc1-303f060864a8\" (UID: \"0dda64d8-2cc8-4164-afc1-303f060864a8\") "
Jul 28 20:00:55.427217 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:55.427096    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0dda64d8-2cc8-4164-afc1-303f060864a8-bound-sa-token\") pod \"0dda64d8-2cc8-4164-afc1-303f060864a8\" (UID: \"0dda64d8-2cc8-4164-afc1-303f060864a8\") "
Jul 28 20:00:55.427217 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:55.427125    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ccwd5\" (UniqueName: \"kubernetes.io/projected/0dda64d8-2cc8-4164-afc1-303f060864a8-kube-api-access-ccwd5\") pod \"0dda64d8-2cc8-4164-afc1-303f060864a8\" (UID: \"0dda64d8-2cc8-4164-afc1-303f060864a8\") "
Jul 28 20:00:55.427217 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:55.427175    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0dda64d8-2cc8-4164-afc1-303f060864a8-installation-pull-secrets\") pod \"0dda64d8-2cc8-4164-afc1-303f060864a8\" (UID: \"0dda64d8-2cc8-4164-afc1-303f060864a8\") "
Jul 28 20:00:55.427217 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:55.427204    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0dda64d8-2cc8-4164-afc1-303f060864a8-registry-tls\") pod \"0dda64d8-2cc8-4164-afc1-303f060864a8\" (UID: \"0dda64d8-2cc8-4164-afc1-303f060864a8\") "
Jul 28 20:00:55.427482 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:55.427231    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0dda64d8-2cc8-4164-afc1-303f060864a8-registry-certificates\") pod \"0dda64d8-2cc8-4164-afc1-303f060864a8\" (UID: \"0dda64d8-2cc8-4164-afc1-303f060864a8\") "
Jul 28 20:00:55.427482 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:55.427262    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0dda64d8-2cc8-4164-afc1-303f060864a8-image-registry-private-configuration\") pod \"0dda64d8-2cc8-4164-afc1-303f060864a8\" (UID: \"0dda64d8-2cc8-4164-afc1-303f060864a8\") "
Jul 28 20:00:55.427578 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:55.427488    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0dda64d8-2cc8-4164-afc1-303f060864a8-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "0dda64d8-2cc8-4164-afc1-303f060864a8" (UID: "0dda64d8-2cc8-4164-afc1-303f060864a8"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:00:55.427738 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:55.427711    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0dda64d8-2cc8-4164-afc1-303f060864a8-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "0dda64d8-2cc8-4164-afc1-303f060864a8" (UID: "0dda64d8-2cc8-4164-afc1-303f060864a8"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:00:55.429811 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:55.429779    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0dda64d8-2cc8-4164-afc1-303f060864a8-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "0dda64d8-2cc8-4164-afc1-303f060864a8" (UID: "0dda64d8-2cc8-4164-afc1-303f060864a8"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:00:55.429996 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:55.429978    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0dda64d8-2cc8-4164-afc1-303f060864a8-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "0dda64d8-2cc8-4164-afc1-303f060864a8" (UID: "0dda64d8-2cc8-4164-afc1-303f060864a8"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:00:55.430104 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:55.430037    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0dda64d8-2cc8-4164-afc1-303f060864a8-kube-api-access-ccwd5" (OuterVolumeSpecName: "kube-api-access-ccwd5") pod "0dda64d8-2cc8-4164-afc1-303f060864a8" (UID: "0dda64d8-2cc8-4164-afc1-303f060864a8"). InnerVolumeSpecName "kube-api-access-ccwd5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:00:55.430167 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:55.430089    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0dda64d8-2cc8-4164-afc1-303f060864a8-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "0dda64d8-2cc8-4164-afc1-303f060864a8" (UID: "0dda64d8-2cc8-4164-afc1-303f060864a8"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:00:55.430206 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:55.430156    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0dda64d8-2cc8-4164-afc1-303f060864a8-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "0dda64d8-2cc8-4164-afc1-303f060864a8" (UID: "0dda64d8-2cc8-4164-afc1-303f060864a8"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:00:55.437816 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:55.437787    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0dda64d8-2cc8-4164-afc1-303f060864a8-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "0dda64d8-2cc8-4164-afc1-303f060864a8" (UID: "0dda64d8-2cc8-4164-afc1-303f060864a8"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:00:55.528353 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:55.528262    2554 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0dda64d8-2cc8-4164-afc1-303f060864a8-trusted-ca\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:00:55.528353 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:55.528309    2554 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0dda64d8-2cc8-4164-afc1-303f060864a8-bound-sa-token\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:00:55.528353 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:55.528330    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ccwd5\" (UniqueName: \"kubernetes.io/projected/0dda64d8-2cc8-4164-afc1-303f060864a8-kube-api-access-ccwd5\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:00:55.528353 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:55.528351    2554 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0dda64d8-2cc8-4164-afc1-303f060864a8-installation-pull-secrets\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:00:55.528581 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:55.528371    2554 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0dda64d8-2cc8-4164-afc1-303f060864a8-registry-tls\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:00:55.528581 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:55.528394    2554 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0dda64d8-2cc8-4164-afc1-303f060864a8-registry-certificates\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:00:55.528581 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:55.528414    2554 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0dda64d8-2cc8-4164-afc1-303f060864a8-image-registry-private-configuration\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:00:55.528581 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:55.528432    2554 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0dda64d8-2cc8-4164-afc1-303f060864a8-ca-trust-extracted\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:00:55.535760 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:55.535725    2554 generic.go:356] "Generic (PLEG): container finished" podID="0dda64d8-2cc8-4164-afc1-303f060864a8" containerID="689dd5e43697577273194f0e783bec3663c45a09d135cc2409c0279f46ec676f" exitCode=0
Jul 28 20:00:55.535943 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:55.535783    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7bcc689c88-gmt7f"
Jul 28 20:00:55.535943 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:55.535812    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7bcc689c88-gmt7f" event={"ID":"0dda64d8-2cc8-4164-afc1-303f060864a8","Type":"ContainerDied","Data":"689dd5e43697577273194f0e783bec3663c45a09d135cc2409c0279f46ec676f"}
Jul 28 20:00:55.535943 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:55.535898    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7bcc689c88-gmt7f" event={"ID":"0dda64d8-2cc8-4164-afc1-303f060864a8","Type":"ContainerDied","Data":"96c1ae50457d724a8087dec30a5c1b8d2928e6e3c6ec5804ddadcec21aa4914f"}
Jul 28 20:00:55.535943 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:55.535927    2554 scope.go:117] "RemoveContainer" containerID="689dd5e43697577273194f0e783bec3663c45a09d135cc2409c0279f46ec676f"
Jul 28 20:00:55.544862 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:55.544845    2554 scope.go:117] "RemoveContainer" containerID="689dd5e43697577273194f0e783bec3663c45a09d135cc2409c0279f46ec676f"
Jul 28 20:00:55.545200 ip-10-0-139-54 kubenswrapper[2554]: E0728 20:00:55.545170    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"689dd5e43697577273194f0e783bec3663c45a09d135cc2409c0279f46ec676f\": container with ID starting with 689dd5e43697577273194f0e783bec3663c45a09d135cc2409c0279f46ec676f not found: ID does not exist" containerID="689dd5e43697577273194f0e783bec3663c45a09d135cc2409c0279f46ec676f"
Jul 28 20:00:55.545286 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:55.545211    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"689dd5e43697577273194f0e783bec3663c45a09d135cc2409c0279f46ec676f"} err="failed to get container status \"689dd5e43697577273194f0e783bec3663c45a09d135cc2409c0279f46ec676f\": rpc error: code = NotFound desc = could not find container \"689dd5e43697577273194f0e783bec3663c45a09d135cc2409c0279f46ec676f\": container with ID starting with 689dd5e43697577273194f0e783bec3663c45a09d135cc2409c0279f46ec676f not found: ID does not exist"
Jul 28 20:00:55.558270 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:55.558233    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7bcc689c88-gmt7f"]
Jul 28 20:00:55.561683 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:55.561659    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7bcc689c88-gmt7f"]
Jul 28 20:00:55.776551 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:00:55.776517    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0dda64d8-2cc8-4164-afc1-303f060864a8" path="/var/lib/kubelet/pods/0dda64d8-2cc8-4164-afc1-303f060864a8/volumes"
Jul 28 20:01:09.589109 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:09.589056    2554 generic.go:356] "Generic (PLEG): container finished" podID="26b203ca-0b58-4389-ac85-8fdc909e11d2" containerID="56554e28cf207950fdb3c05131dc9c0ca57599719f163ea72fadec35ffe84551" exitCode=0
Jul 28 20:01:09.589478 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:09.589128    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-mfvfs" event={"ID":"26b203ca-0b58-4389-ac85-8fdc909e11d2","Type":"ContainerDied","Data":"56554e28cf207950fdb3c05131dc9c0ca57599719f163ea72fadec35ffe84551"}
Jul 28 20:01:09.589478 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:09.589458    2554 scope.go:117] "RemoveContainer" containerID="56554e28cf207950fdb3c05131dc9c0ca57599719f163ea72fadec35ffe84551"
Jul 28 20:01:10.594691 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:10.594658    2554 generic.go:356] "Generic (PLEG): container finished" podID="2d7a85b0-e81f-4686-8889-5096efe82a1a" containerID="2c90070ca04436e38308b48a03a960139652b828a6a0916588f7bbcee7fcd86a" exitCode=0
Jul 28 20:01:10.595135 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:10.594733    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-xjfzb" event={"ID":"2d7a85b0-e81f-4686-8889-5096efe82a1a","Type":"ContainerDied","Data":"2c90070ca04436e38308b48a03a960139652b828a6a0916588f7bbcee7fcd86a"}
Jul 28 20:01:10.595215 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:10.595154    2554 scope.go:117] "RemoveContainer" containerID="2c90070ca04436e38308b48a03a960139652b828a6a0916588f7bbcee7fcd86a"
Jul 28 20:01:10.596633 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:10.596612    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-mfvfs" event={"ID":"26b203ca-0b58-4389-ac85-8fdc909e11d2","Type":"ContainerStarted","Data":"c05ded959b4af8cb1615e16e118e85ea5f9c62b609e6dacc51a57bef062d175f"}
Jul 28 20:01:11.602673 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:11.602636    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-xjfzb" event={"ID":"2d7a85b0-e81f-4686-8889-5096efe82a1a","Type":"ContainerStarted","Data":"d9f2f357ee640c556137c5a54da1f4c667794b7f472cb51107aa149245fe5dfa"}
Jul 28 20:01:15.620803 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:15.620769    2554 generic.go:356] "Generic (PLEG): container finished" podID="8f22d663-ad11-402a-a2a6-2f66e778b4ff" containerID="49de1ce79e48987391516775f94530bd30c2b97c2f0eb13e95972065395bc557" exitCode=0
Jul 28 20:01:15.621244 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:15.620837    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-k9b95" event={"ID":"8f22d663-ad11-402a-a2a6-2f66e778b4ff","Type":"ContainerDied","Data":"49de1ce79e48987391516775f94530bd30c2b97c2f0eb13e95972065395bc557"}
Jul 28 20:01:15.621244 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:15.621203    2554 scope.go:117] "RemoveContainer" containerID="49de1ce79e48987391516775f94530bd30c2b97c2f0eb13e95972065395bc557"
Jul 28 20:01:16.626464 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:16.626430    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-k9b95" event={"ID":"8f22d663-ad11-402a-a2a6-2f66e778b4ff","Type":"ContainerStarted","Data":"880719d27420a7d27303c45aeb49c09631d0ef12aa559a6349cc64095d04162d"}
Jul 28 20:01:39.478808 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:39.478752    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:39.494474 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:39.494449    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:39.729420 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:39.729344    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:57.547420 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.547342    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 20:01:57.547897 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.547717    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="07c3bf7b-8b01-422a-ac70-04aa061e488f" containerName="prometheus" containerID="cri-o://d841b38735c03b304249adb9640c46d374a668692f3128b6d7540bcde592e4ea" gracePeriod=600
Jul 28 20:01:57.547897 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.547724    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="07c3bf7b-8b01-422a-ac70-04aa061e488f" containerName="kube-rbac-proxy-web" containerID="cri-o://80aa6654c5840b30f1b786b81956267cf43ee950fac48b8b48d8fb1dd20bb7f2" gracePeriod=600
Jul 28 20:01:57.547897 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.547725    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="07c3bf7b-8b01-422a-ac70-04aa061e488f" containerName="kube-rbac-proxy" containerID="cri-o://b0b4ad53db8bb0123338df8dbb727d4bae7e17875d5551bccbce138c8c36fbf4" gracePeriod=600
Jul 28 20:01:57.547897 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.547765    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="07c3bf7b-8b01-422a-ac70-04aa061e488f" containerName="config-reloader" containerID="cri-o://33ba837ca10f256b80049701ee6944b1a0612b2ec5b48d57941bdff4ce730763" gracePeriod=600
Jul 28 20:01:57.547897 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.547827    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="07c3bf7b-8b01-422a-ac70-04aa061e488f" containerName="kube-rbac-proxy-thanos" containerID="cri-o://c8230091c31c7f2421b1227b5bcdfd8522650f665721c7763cb157163888f113" gracePeriod=600
Jul 28 20:01:57.547897 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.547759    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="07c3bf7b-8b01-422a-ac70-04aa061e488f" containerName="thanos-sidecar" containerID="cri-o://b4031d15d133ac43134fafeb273f03afedd787f1cd321bc6ca36c19ba2d13d0c" gracePeriod=600
Jul 28 20:01:57.782501 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.782475    2554 generic.go:356] "Generic (PLEG): container finished" podID="07c3bf7b-8b01-422a-ac70-04aa061e488f" containerID="c8230091c31c7f2421b1227b5bcdfd8522650f665721c7763cb157163888f113" exitCode=0
Jul 28 20:01:57.782501 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.782497    2554 generic.go:356] "Generic (PLEG): container finished" podID="07c3bf7b-8b01-422a-ac70-04aa061e488f" containerID="b0b4ad53db8bb0123338df8dbb727d4bae7e17875d5551bccbce138c8c36fbf4" exitCode=0
Jul 28 20:01:57.782501 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.782504    2554 generic.go:356] "Generic (PLEG): container finished" podID="07c3bf7b-8b01-422a-ac70-04aa061e488f" containerID="80aa6654c5840b30f1b786b81956267cf43ee950fac48b8b48d8fb1dd20bb7f2" exitCode=0
Jul 28 20:01:57.782501 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.782509    2554 generic.go:356] "Generic (PLEG): container finished" podID="07c3bf7b-8b01-422a-ac70-04aa061e488f" containerID="b4031d15d133ac43134fafeb273f03afedd787f1cd321bc6ca36c19ba2d13d0c" exitCode=0
Jul 28 20:01:57.782809 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.782515    2554 generic.go:356] "Generic (PLEG): container finished" podID="07c3bf7b-8b01-422a-ac70-04aa061e488f" containerID="33ba837ca10f256b80049701ee6944b1a0612b2ec5b48d57941bdff4ce730763" exitCode=0
Jul 28 20:01:57.782809 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.782524    2554 generic.go:356] "Generic (PLEG): container finished" podID="07c3bf7b-8b01-422a-ac70-04aa061e488f" containerID="d841b38735c03b304249adb9640c46d374a668692f3128b6d7540bcde592e4ea" exitCode=0
Jul 28 20:01:57.782809 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.782540    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"07c3bf7b-8b01-422a-ac70-04aa061e488f","Type":"ContainerDied","Data":"c8230091c31c7f2421b1227b5bcdfd8522650f665721c7763cb157163888f113"}
Jul 28 20:01:57.782809 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.782612    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"07c3bf7b-8b01-422a-ac70-04aa061e488f","Type":"ContainerDied","Data":"b0b4ad53db8bb0123338df8dbb727d4bae7e17875d5551bccbce138c8c36fbf4"}
Jul 28 20:01:57.782809 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.782649    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"07c3bf7b-8b01-422a-ac70-04aa061e488f","Type":"ContainerDied","Data":"80aa6654c5840b30f1b786b81956267cf43ee950fac48b8b48d8fb1dd20bb7f2"}
Jul 28 20:01:57.782809 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.782674    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"07c3bf7b-8b01-422a-ac70-04aa061e488f","Type":"ContainerDied","Data":"b4031d15d133ac43134fafeb273f03afedd787f1cd321bc6ca36c19ba2d13d0c"}
Jul 28 20:01:57.782809 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.782705    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"07c3bf7b-8b01-422a-ac70-04aa061e488f","Type":"ContainerDied","Data":"33ba837ca10f256b80049701ee6944b1a0612b2ec5b48d57941bdff4ce730763"}
Jul 28 20:01:57.782809 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.782735    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"07c3bf7b-8b01-422a-ac70-04aa061e488f","Type":"ContainerDied","Data":"d841b38735c03b304249adb9640c46d374a668692f3128b6d7540bcde592e4ea"}
Jul 28 20:01:57.789191 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.789170    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:57.869496 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.869416    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/07c3bf7b-8b01-422a-ac70-04aa061e488f-prometheus-k8s-rulefiles-0\") pod \"07c3bf7b-8b01-422a-ac70-04aa061e488f\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") "
Jul 28 20:01:57.869496 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.869459    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-config\") pod \"07c3bf7b-8b01-422a-ac70-04aa061e488f\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") "
Jul 28 20:01:57.869695 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.869494    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"07c3bf7b-8b01-422a-ac70-04aa061e488f\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") "
Jul 28 20:01:57.869695 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.869520    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07c3bf7b-8b01-422a-ac70-04aa061e488f-prometheus-trusted-ca-bundle\") pod \"07c3bf7b-8b01-422a-ac70-04aa061e488f\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") "
Jul 28 20:01:57.869695 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.869546    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/07c3bf7b-8b01-422a-ac70-04aa061e488f-tls-assets\") pod \"07c3bf7b-8b01-422a-ac70-04aa061e488f\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") "
Jul 28 20:01:57.869695 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.869575    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/07c3bf7b-8b01-422a-ac70-04aa061e488f-prometheus-k8s-rulefiles-2\") pod \"07c3bf7b-8b01-422a-ac70-04aa061e488f\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") "
Jul 28 20:01:57.869695 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.869601    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r62n8\" (UniqueName: \"kubernetes.io/projected/07c3bf7b-8b01-422a-ac70-04aa061e488f-kube-api-access-r62n8\") pod \"07c3bf7b-8b01-422a-ac70-04aa061e488f\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") "
Jul 28 20:01:57.869695 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.869627    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/07c3bf7b-8b01-422a-ac70-04aa061e488f-prometheus-k8s-db\") pod \"07c3bf7b-8b01-422a-ac70-04aa061e488f\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") "
Jul 28 20:01:57.869695 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.869665    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/07c3bf7b-8b01-422a-ac70-04aa061e488f-configmap-metrics-client-ca\") pod \"07c3bf7b-8b01-422a-ac70-04aa061e488f\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") "
Jul 28 20:01:57.870030 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.869711    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-thanos-prometheus-http-client-file\") pod \"07c3bf7b-8b01-422a-ac70-04aa061e488f\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") "
Jul 28 20:01:57.870030 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.869739    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"07c3bf7b-8b01-422a-ac70-04aa061e488f\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") "
Jul 28 20:01:57.870030 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.869769    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-secret-kube-rbac-proxy\") pod \"07c3bf7b-8b01-422a-ac70-04aa061e488f\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") "
Jul 28 20:01:57.870030 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.869797    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07c3bf7b-8b01-422a-ac70-04aa061e488f-configmap-kubelet-serving-ca-bundle\") pod \"07c3bf7b-8b01-422a-ac70-04aa061e488f\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") "
Jul 28 20:01:57.870030 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.869834    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07c3bf7b-8b01-422a-ac70-04aa061e488f-configmap-serving-certs-ca-bundle\") pod \"07c3bf7b-8b01-422a-ac70-04aa061e488f\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") "
Jul 28 20:01:57.870030 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.869860    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-secret-metrics-client-certs\") pod \"07c3bf7b-8b01-422a-ac70-04aa061e488f\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") "
Jul 28 20:01:57.870030 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.869887    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-secret-grpc-tls\") pod \"07c3bf7b-8b01-422a-ac70-04aa061e488f\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") "
Jul 28 20:01:57.870030 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.869916    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-web-config\") pod \"07c3bf7b-8b01-422a-ac70-04aa061e488f\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") "
Jul 28 20:01:57.870030 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.869914    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07c3bf7b-8b01-422a-ac70-04aa061e488f-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "07c3bf7b-8b01-422a-ac70-04aa061e488f" (UID: "07c3bf7b-8b01-422a-ac70-04aa061e488f"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:01:57.870030 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.869947    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/07c3bf7b-8b01-422a-ac70-04aa061e488f-config-out\") pod \"07c3bf7b-8b01-422a-ac70-04aa061e488f\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") "
Jul 28 20:01:57.870030 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.870012    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-secret-prometheus-k8s-tls\") pod \"07c3bf7b-8b01-422a-ac70-04aa061e488f\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") "
Jul 28 20:01:57.870552 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.870041    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/07c3bf7b-8b01-422a-ac70-04aa061e488f-prometheus-k8s-rulefiles-1\") pod \"07c3bf7b-8b01-422a-ac70-04aa061e488f\" (UID: \"07c3bf7b-8b01-422a-ac70-04aa061e488f\") "
Jul 28 20:01:57.870552 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.870297    2554 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07c3bf7b-8b01-422a-ac70-04aa061e488f-prometheus-trusted-ca-bundle\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:01:57.870552 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.870309    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07c3bf7b-8b01-422a-ac70-04aa061e488f-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "07c3bf7b-8b01-422a-ac70-04aa061e488f" (UID: "07c3bf7b-8b01-422a-ac70-04aa061e488f"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:01:57.870706 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.870651    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07c3bf7b-8b01-422a-ac70-04aa061e488f-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "07c3bf7b-8b01-422a-ac70-04aa061e488f" (UID: "07c3bf7b-8b01-422a-ac70-04aa061e488f"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:01:57.871018 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.870992    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07c3bf7b-8b01-422a-ac70-04aa061e488f-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "07c3bf7b-8b01-422a-ac70-04aa061e488f" (UID: "07c3bf7b-8b01-422a-ac70-04aa061e488f"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:01:57.871193 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.871159    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07c3bf7b-8b01-422a-ac70-04aa061e488f-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "07c3bf7b-8b01-422a-ac70-04aa061e488f" (UID: "07c3bf7b-8b01-422a-ac70-04aa061e488f"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:01:57.872802 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.872596    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "07c3bf7b-8b01-422a-ac70-04aa061e488f" (UID: "07c3bf7b-8b01-422a-ac70-04aa061e488f"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:01:57.873104 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.873059    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07c3bf7b-8b01-422a-ac70-04aa061e488f-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "07c3bf7b-8b01-422a-ac70-04aa061e488f" (UID: "07c3bf7b-8b01-422a-ac70-04aa061e488f"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:01:57.873279 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.873245    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07c3bf7b-8b01-422a-ac70-04aa061e488f-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "07c3bf7b-8b01-422a-ac70-04aa061e488f" (UID: "07c3bf7b-8b01-422a-ac70-04aa061e488f"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:01:57.873659 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.873633    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07c3bf7b-8b01-422a-ac70-04aa061e488f-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "07c3bf7b-8b01-422a-ac70-04aa061e488f" (UID: "07c3bf7b-8b01-422a-ac70-04aa061e488f"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:01:57.873877 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.873849    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "07c3bf7b-8b01-422a-ac70-04aa061e488f" (UID: "07c3bf7b-8b01-422a-ac70-04aa061e488f"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:01:57.874260 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.874168    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "07c3bf7b-8b01-422a-ac70-04aa061e488f" (UID: "07c3bf7b-8b01-422a-ac70-04aa061e488f"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:01:57.874260 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.874215    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-config" (OuterVolumeSpecName: "config") pod "07c3bf7b-8b01-422a-ac70-04aa061e488f" (UID: "07c3bf7b-8b01-422a-ac70-04aa061e488f"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:01:57.874413 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.874315    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "07c3bf7b-8b01-422a-ac70-04aa061e488f" (UID: "07c3bf7b-8b01-422a-ac70-04aa061e488f"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:01:57.874691 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.874666    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07c3bf7b-8b01-422a-ac70-04aa061e488f-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "07c3bf7b-8b01-422a-ac70-04aa061e488f" (UID: "07c3bf7b-8b01-422a-ac70-04aa061e488f"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:01:57.874927 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.874902    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "07c3bf7b-8b01-422a-ac70-04aa061e488f" (UID: "07c3bf7b-8b01-422a-ac70-04aa061e488f"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:01:57.875003 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.874963    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "07c3bf7b-8b01-422a-ac70-04aa061e488f" (UID: "07c3bf7b-8b01-422a-ac70-04aa061e488f"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:01:57.875223 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.875200    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "07c3bf7b-8b01-422a-ac70-04aa061e488f" (UID: "07c3bf7b-8b01-422a-ac70-04aa061e488f"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:01:57.875816 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.875794    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07c3bf7b-8b01-422a-ac70-04aa061e488f-config-out" (OuterVolumeSpecName: "config-out") pod "07c3bf7b-8b01-422a-ac70-04aa061e488f" (UID: "07c3bf7b-8b01-422a-ac70-04aa061e488f"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:01:57.875900 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.875827    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07c3bf7b-8b01-422a-ac70-04aa061e488f-kube-api-access-r62n8" (OuterVolumeSpecName: "kube-api-access-r62n8") pod "07c3bf7b-8b01-422a-ac70-04aa061e488f" (UID: "07c3bf7b-8b01-422a-ac70-04aa061e488f"). InnerVolumeSpecName "kube-api-access-r62n8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:01:57.884430 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.884402    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-web-config" (OuterVolumeSpecName: "web-config") pod "07c3bf7b-8b01-422a-ac70-04aa061e488f" (UID: "07c3bf7b-8b01-422a-ac70-04aa061e488f"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:01:57.971654 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.971616    2554 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-secret-kube-rbac-proxy\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:01:57.971654 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.971650    2554 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07c3bf7b-8b01-422a-ac70-04aa061e488f-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:01:57.971654 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.971661    2554 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07c3bf7b-8b01-422a-ac70-04aa061e488f-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:01:57.971862 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.971671    2554 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-secret-metrics-client-certs\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:01:57.971862 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.971680    2554 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-secret-grpc-tls\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:01:57.971862 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.971710    2554 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-web-config\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:01:57.971862 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.971718    2554 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/07c3bf7b-8b01-422a-ac70-04aa061e488f-config-out\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:01:57.971862 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.971726    2554 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-secret-prometheus-k8s-tls\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:01:57.971862 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.971737    2554 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/07c3bf7b-8b01-422a-ac70-04aa061e488f-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:01:57.971862 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.971745    2554 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/07c3bf7b-8b01-422a-ac70-04aa061e488f-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:01:57.971862 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.971753    2554 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-config\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:01:57.971862 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.971762    2554 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:01:57.971862 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.971771    2554 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/07c3bf7b-8b01-422a-ac70-04aa061e488f-tls-assets\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:01:57.971862 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.971779    2554 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/07c3bf7b-8b01-422a-ac70-04aa061e488f-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:01:57.971862 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.971787    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r62n8\" (UniqueName: \"kubernetes.io/projected/07c3bf7b-8b01-422a-ac70-04aa061e488f-kube-api-access-r62n8\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:01:57.971862 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.971795    2554 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/07c3bf7b-8b01-422a-ac70-04aa061e488f-prometheus-k8s-db\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:01:57.971862 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.971804    2554 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/07c3bf7b-8b01-422a-ac70-04aa061e488f-configmap-metrics-client-ca\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:01:57.971862 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.971812    2554 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-thanos-prometheus-http-client-file\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:01:57.971862 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:57.971821    2554 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/07c3bf7b-8b01-422a-ac70-04aa061e488f-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:01:58.789540 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.789503    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"07c3bf7b-8b01-422a-ac70-04aa061e488f","Type":"ContainerDied","Data":"c64f499107984111fd92900a9da8c196b7c3175115ebfb089065aa23c213d7ff"}
Jul 28 20:01:58.790001 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.789571    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:58.790001 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.789582    2554 scope.go:117] "RemoveContainer" containerID="c8230091c31c7f2421b1227b5bcdfd8522650f665721c7763cb157163888f113"
Jul 28 20:01:58.798210 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.798193    2554 scope.go:117] "RemoveContainer" containerID="b0b4ad53db8bb0123338df8dbb727d4bae7e17875d5551bccbce138c8c36fbf4"
Jul 28 20:01:58.805319 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.805300    2554 scope.go:117] "RemoveContainer" containerID="80aa6654c5840b30f1b786b81956267cf43ee950fac48b8b48d8fb1dd20bb7f2"
Jul 28 20:01:58.811995 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.811976    2554 scope.go:117] "RemoveContainer" containerID="b4031d15d133ac43134fafeb273f03afedd787f1cd321bc6ca36c19ba2d13d0c"
Jul 28 20:01:58.818503 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.818482    2554 scope.go:117] "RemoveContainer" containerID="33ba837ca10f256b80049701ee6944b1a0612b2ec5b48d57941bdff4ce730763"
Jul 28 20:01:58.822342 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.822317    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 20:01:58.825408 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.825393    2554 scope.go:117] "RemoveContainer" containerID="d841b38735c03b304249adb9640c46d374a668692f3128b6d7540bcde592e4ea"
Jul 28 20:01:58.828517 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.828492    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 20:01:58.832867 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.832851    2554 scope.go:117] "RemoveContainer" containerID="a633dbf7a8f980361a74363cda62edec760ccd9ed470b4464fb1bdc423090fb9"
Jul 28 20:01:58.870211 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.870177    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 20:01:58.870453 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.870442    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07c3bf7b-8b01-422a-ac70-04aa061e488f" containerName="prometheus"
Jul 28 20:01:58.870540 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.870455    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="07c3bf7b-8b01-422a-ac70-04aa061e488f" containerName="prometheus"
Jul 28 20:01:58.870540 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.870467    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07c3bf7b-8b01-422a-ac70-04aa061e488f" containerName="kube-rbac-proxy-thanos"
Jul 28 20:01:58.870540 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.870473    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="07c3bf7b-8b01-422a-ac70-04aa061e488f" containerName="kube-rbac-proxy-thanos"
Jul 28 20:01:58.870540 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.870483    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0dda64d8-2cc8-4164-afc1-303f060864a8" containerName="registry"
Jul 28 20:01:58.870540 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.870489    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="0dda64d8-2cc8-4164-afc1-303f060864a8" containerName="registry"
Jul 28 20:01:58.870540 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.870495    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07c3bf7b-8b01-422a-ac70-04aa061e488f" containerName="init-config-reloader"
Jul 28 20:01:58.870540 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.870500    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="07c3bf7b-8b01-422a-ac70-04aa061e488f" containerName="init-config-reloader"
Jul 28 20:01:58.870540 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.870507    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07c3bf7b-8b01-422a-ac70-04aa061e488f" containerName="kube-rbac-proxy-web"
Jul 28 20:01:58.870540 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.870512    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="07c3bf7b-8b01-422a-ac70-04aa061e488f" containerName="kube-rbac-proxy-web"
Jul 28 20:01:58.870540 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.870519    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07c3bf7b-8b01-422a-ac70-04aa061e488f" containerName="config-reloader"
Jul 28 20:01:58.870540 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.870525    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="07c3bf7b-8b01-422a-ac70-04aa061e488f" containerName="config-reloader"
Jul 28 20:01:58.870540 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.870533    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07c3bf7b-8b01-422a-ac70-04aa061e488f" containerName="thanos-sidecar"
Jul 28 20:01:58.870540 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.870537    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="07c3bf7b-8b01-422a-ac70-04aa061e488f" containerName="thanos-sidecar"
Jul 28 20:01:58.870893 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.870548    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07c3bf7b-8b01-422a-ac70-04aa061e488f" containerName="kube-rbac-proxy"
Jul 28 20:01:58.870893 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.870553    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="07c3bf7b-8b01-422a-ac70-04aa061e488f" containerName="kube-rbac-proxy"
Jul 28 20:01:58.870893 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.870593    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="0dda64d8-2cc8-4164-afc1-303f060864a8" containerName="registry"
Jul 28 20:01:58.870893 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.870599    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="07c3bf7b-8b01-422a-ac70-04aa061e488f" containerName="prometheus"
Jul 28 20:01:58.870893 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.870605    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="07c3bf7b-8b01-422a-ac70-04aa061e488f" containerName="config-reloader"
Jul 28 20:01:58.870893 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.870611    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="07c3bf7b-8b01-422a-ac70-04aa061e488f" containerName="thanos-sidecar"
Jul 28 20:01:58.870893 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.870615    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="07c3bf7b-8b01-422a-ac70-04aa061e488f" containerName="kube-rbac-proxy-web"
Jul 28 20:01:58.870893 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.870620    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="07c3bf7b-8b01-422a-ac70-04aa061e488f" containerName="kube-rbac-proxy"
Jul 28 20:01:58.870893 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.870625    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="07c3bf7b-8b01-422a-ac70-04aa061e488f" containerName="kube-rbac-proxy-thanos"
Jul 28 20:01:58.874403 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.874385    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:58.883045 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.883009    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 28 20:01:58.883191 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.883012    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 28 20:01:58.883598 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.883568    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 28 20:01:58.883695 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.883641    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 28 20:01:58.883859 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.883844    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 28 20:01:58.883966 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.883945    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 28 20:01:58.884099 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.884059    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-6trmh4jod19jl\""
Jul 28 20:01:58.884236 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.884223    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 28 20:01:58.885137 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.885107    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 28 20:01:58.888057 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.888036    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-p9hzm\""
Jul 28 20:01:58.888172 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.888057    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 28 20:01:58.888172 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.888046    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 28 20:01:58.888172 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.888040    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 28 20:01:58.888172 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.888163    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 28 20:01:58.888339 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.888268    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 28 20:01:58.888472 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.888457    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 28 20:01:58.888895 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.888870    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 28 20:01:58.901753 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.901727    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 20:01:58.980847 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.980810    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/60850744-b39b-4b11-af79-d5baac9854c0-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:58.980847 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.980845    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/60850744-b39b-4b11-af79-d5baac9854c0-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:58.981125 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.980868    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/60850744-b39b-4b11-af79-d5baac9854c0-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:58.981125 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.980906    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/60850744-b39b-4b11-af79-d5baac9854c0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:58.981125 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.980943    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/60850744-b39b-4b11-af79-d5baac9854c0-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:58.981125 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.981016    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/60850744-b39b-4b11-af79-d5baac9854c0-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:58.981125 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.981091    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/60850744-b39b-4b11-af79-d5baac9854c0-config\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:58.981125 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.981127    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/60850744-b39b-4b11-af79-d5baac9854c0-web-config\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:58.981373 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.981157    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/60850744-b39b-4b11-af79-d5baac9854c0-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:58.981373 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.981187    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60850744-b39b-4b11-af79-d5baac9854c0-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:58.981373 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.981230    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60850744-b39b-4b11-af79-d5baac9854c0-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:58.981373 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.981271    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/60850744-b39b-4b11-af79-d5baac9854c0-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:58.981373 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.981308    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/60850744-b39b-4b11-af79-d5baac9854c0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:58.981373 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.981335    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c49gw\" (UniqueName: \"kubernetes.io/projected/60850744-b39b-4b11-af79-d5baac9854c0-kube-api-access-c49gw\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:58.981373 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.981351    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/60850744-b39b-4b11-af79-d5baac9854c0-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:58.981373 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.981370    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/60850744-b39b-4b11-af79-d5baac9854c0-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:58.981636 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.981385    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/60850744-b39b-4b11-af79-d5baac9854c0-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:58.981636 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.981451    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60850744-b39b-4b11-af79-d5baac9854c0-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:58.981636 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.981477    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/60850744-b39b-4b11-af79-d5baac9854c0-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:58.981636 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:58.981504    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/60850744-b39b-4b11-af79-d5baac9854c0-config-out\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:59.082196 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:59.082106    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/60850744-b39b-4b11-af79-d5baac9854c0-web-config\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:59.082196 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:59.082151    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/60850744-b39b-4b11-af79-d5baac9854c0-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:59.082196 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:59.082169    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60850744-b39b-4b11-af79-d5baac9854c0-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:59.082196 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:59.082186    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60850744-b39b-4b11-af79-d5baac9854c0-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:59.082196 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:59.082203    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/60850744-b39b-4b11-af79-d5baac9854c0-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:59.082541 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:59.082228    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/60850744-b39b-4b11-af79-d5baac9854c0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:59.082541 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:59.082252    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c49gw\" (UniqueName: \"kubernetes.io/projected/60850744-b39b-4b11-af79-d5baac9854c0-kube-api-access-c49gw\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:59.082541 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:59.082269    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/60850744-b39b-4b11-af79-d5baac9854c0-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:59.082541 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:59.082299    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/60850744-b39b-4b11-af79-d5baac9854c0-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:59.082541 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:59.082325    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/60850744-b39b-4b11-af79-d5baac9854c0-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:59.082541 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:59.082360    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60850744-b39b-4b11-af79-d5baac9854c0-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:59.083049 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:59.083019    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/60850744-b39b-4b11-af79-d5baac9854c0-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:59.083857 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:59.083098    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/60850744-b39b-4b11-af79-d5baac9854c0-config-out\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:59.083857 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:59.083139    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/60850744-b39b-4b11-af79-d5baac9854c0-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:59.083857 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:59.083157    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60850744-b39b-4b11-af79-d5baac9854c0-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:59.083857 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:59.083169    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/60850744-b39b-4b11-af79-d5baac9854c0-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:59.083857 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:59.083191    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60850744-b39b-4b11-af79-d5baac9854c0-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:59.083857 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:59.083190    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/60850744-b39b-4b11-af79-d5baac9854c0-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:59.083857 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:59.083319    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/60850744-b39b-4b11-af79-d5baac9854c0-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:59.083857 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:59.083354    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/60850744-b39b-4b11-af79-d5baac9854c0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:59.083857 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:59.083387    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/60850744-b39b-4b11-af79-d5baac9854c0-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:59.083857 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:59.083425    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/60850744-b39b-4b11-af79-d5baac9854c0-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:59.083857 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:59.083460    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/60850744-b39b-4b11-af79-d5baac9854c0-config\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:59.083857 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:59.083470    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/60850744-b39b-4b11-af79-d5baac9854c0-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:59.085744 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:59.084043    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/60850744-b39b-4b11-af79-d5baac9854c0-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:59.085744 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:59.084554    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/60850744-b39b-4b11-af79-d5baac9854c0-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:59.085744 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:59.085439    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/60850744-b39b-4b11-af79-d5baac9854c0-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:59.085744 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:59.085612    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60850744-b39b-4b11-af79-d5baac9854c0-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:59.086137 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:59.086043    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/60850744-b39b-4b11-af79-d5baac9854c0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:59.086137 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:59.086114    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/60850744-b39b-4b11-af79-d5baac9854c0-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:59.086743 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:59.086720    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/60850744-b39b-4b11-af79-d5baac9854c0-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:59.087826 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:59.087619    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/60850744-b39b-4b11-af79-d5baac9854c0-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:59.087826 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:59.087768    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/60850744-b39b-4b11-af79-d5baac9854c0-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:59.087826 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:59.087818    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/60850744-b39b-4b11-af79-d5baac9854c0-config-out\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:59.088039 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:59.087887    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/60850744-b39b-4b11-af79-d5baac9854c0-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:59.088252 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:59.088233    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/60850744-b39b-4b11-af79-d5baac9854c0-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:59.088404 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:59.088381    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/60850744-b39b-4b11-af79-d5baac9854c0-web-config\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:59.088437 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:59.088393    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/60850744-b39b-4b11-af79-d5baac9854c0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:59.089020 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:59.088996    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/60850744-b39b-4b11-af79-d5baac9854c0-config\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:59.091966 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:59.091945    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c49gw\" (UniqueName: \"kubernetes.io/projected/60850744-b39b-4b11-af79-d5baac9854c0-kube-api-access-c49gw\") pod \"prometheus-k8s-0\" (UID: \"60850744-b39b-4b11-af79-d5baac9854c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:59.188911 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:59.188876    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:01:59.329795 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:59.329758    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 28 20:01:59.334029 ip-10-0-139-54 kubenswrapper[2554]: W0728 20:01:59.333971    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60850744_b39b_4b11_af79_d5baac9854c0.slice/crio-539b05c647ce793c66c3d346f4b7436c74b6dd497567189655ba6b6359ea4771 WatchSource:0}: Error finding container 539b05c647ce793c66c3d346f4b7436c74b6dd497567189655ba6b6359ea4771: Status 404 returned error can't find the container with id 539b05c647ce793c66c3d346f4b7436c74b6dd497567189655ba6b6359ea4771
Jul 28 20:01:59.776536 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:59.776455    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07c3bf7b-8b01-422a-ac70-04aa061e488f" path="/var/lib/kubelet/pods/07c3bf7b-8b01-422a-ac70-04aa061e488f/volumes"
Jul 28 20:01:59.794777 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:59.794733    2554 generic.go:356] "Generic (PLEG): container finished" podID="60850744-b39b-4b11-af79-d5baac9854c0" containerID="c11e5ffb944188c8c3577fe8a06383437fb4a623ebd79c2b4a66973d836185be" exitCode=0
Jul 28 20:01:59.795139 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:59.794793    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"60850744-b39b-4b11-af79-d5baac9854c0","Type":"ContainerDied","Data":"c11e5ffb944188c8c3577fe8a06383437fb4a623ebd79c2b4a66973d836185be"}
Jul 28 20:01:59.795139 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:01:59.794846    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"60850744-b39b-4b11-af79-d5baac9854c0","Type":"ContainerStarted","Data":"539b05c647ce793c66c3d346f4b7436c74b6dd497567189655ba6b6359ea4771"}
Jul 28 20:02:00.803641 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:02:00.803605    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"60850744-b39b-4b11-af79-d5baac9854c0","Type":"ContainerStarted","Data":"22455ae80717f115c7a3784b030f973b31deae6f88220595ea26c449f73c31a3"}
Jul 28 20:02:00.804021 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:02:00.803669    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"60850744-b39b-4b11-af79-d5baac9854c0","Type":"ContainerStarted","Data":"98d510c521470bb3c4380d4e1d8f17a9f485d3a2e51d989bab1fb8881ae27648"}
Jul 28 20:02:00.804021 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:02:00.803702    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"60850744-b39b-4b11-af79-d5baac9854c0","Type":"ContainerStarted","Data":"3ab4a9aabc706317d68e5c13a59c3b901cdaa18eb887839c068b7ecf84d209e9"}
Jul 28 20:02:00.804021 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:02:00.803729    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"60850744-b39b-4b11-af79-d5baac9854c0","Type":"ContainerStarted","Data":"e12ec45accda63aabca7664c7ed720c847189a8b5a9415b540cb2aa38ca3e135"}
Jul 28 20:02:00.804021 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:02:00.803760    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"60850744-b39b-4b11-af79-d5baac9854c0","Type":"ContainerStarted","Data":"8a913eb52009555e33f3623d52aae73381bfc426dc2964fb7b878a1c8d7b6883"}
Jul 28 20:02:00.804021 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:02:00.803783    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"60850744-b39b-4b11-af79-d5baac9854c0","Type":"ContainerStarted","Data":"2cf00e7e5abfc1f6e62a6b96b7f23af84907a46eb93a642fa6c86eb96d927d46"}
Jul 28 20:02:04.189812 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:02:04.189771    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:02:59.189703 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:02:59.189668    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:02:59.204894 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:02:59.204863    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:02:59.237853 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:02:59.237594    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=61.218709619 podStartE2EDuration="1m1.237572675s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.863056ms" isStatefulPod=false podCreationTimestamp="2026-07-28 20:01:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:02:00.85131836 +0000 UTC m=+183.634329804" watchObservedRunningTime="2026-07-28 20:02:59.237572675 +0000 UTC m=+242.020584119"
Jul 28 20:03:00.039787 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:03:00.039760    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 28 20:03:57.680923 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:03:57.680889    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-7vcr8_dde4c1a8-8229-4419-87b6-d8e39a0827c4/console-operator/1.log"
Jul 28 20:03:57.685111 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:03:57.685090    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-7vcr8_dde4c1a8-8229-4419-87b6-d8e39a0827c4/console-operator/1.log"
Jul 28 20:03:57.690121 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:03:57.690101    2554 kubelet.go:1648] "Image garbage collection succeeded"
Jul 28 20:05:22.754383 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:22.754300    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-789bdb4f85-rfnwn"]
Jul 28 20:05:22.756750 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:22.756733    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-789bdb4f85-rfnwn"
Jul 28 20:05:22.760320 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:22.760297    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 28 20:05:22.761106 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:22.761084    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 28 20:05:22.761581 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:22.761553    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 28 20:05:22.761581 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:22.761575    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 28 20:05:22.761753 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:22.761575    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 28 20:05:22.761753 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:22.761584    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-48j6r\""
Jul 28 20:05:22.769171 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:22.769147    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-789bdb4f85-rfnwn"]
Jul 28 20:05:22.797334 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:22.797298    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ec7bd9e6-90c1-468b-ba24-d274f070e2b9-manager-config\") pod \"lws-controller-manager-789bdb4f85-rfnwn\" (UID: \"ec7bd9e6-90c1-468b-ba24-d274f070e2b9\") " pod="openshift-lws-operator/lws-controller-manager-789bdb4f85-rfnwn"
Jul 28 20:05:22.797515 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:22.797345    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ec7bd9e6-90c1-468b-ba24-d274f070e2b9-metrics-cert\") pod \"lws-controller-manager-789bdb4f85-rfnwn\" (UID: \"ec7bd9e6-90c1-468b-ba24-d274f070e2b9\") " pod="openshift-lws-operator/lws-controller-manager-789bdb4f85-rfnwn"
Jul 28 20:05:22.797515 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:22.797367    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zl9gp\" (UniqueName: \"kubernetes.io/projected/ec7bd9e6-90c1-468b-ba24-d274f070e2b9-kube-api-access-zl9gp\") pod \"lws-controller-manager-789bdb4f85-rfnwn\" (UID: \"ec7bd9e6-90c1-468b-ba24-d274f070e2b9\") " pod="openshift-lws-operator/lws-controller-manager-789bdb4f85-rfnwn"
Jul 28 20:05:22.797515 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:22.797427    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ec7bd9e6-90c1-468b-ba24-d274f070e2b9-cert\") pod \"lws-controller-manager-789bdb4f85-rfnwn\" (UID: \"ec7bd9e6-90c1-468b-ba24-d274f070e2b9\") " pod="openshift-lws-operator/lws-controller-manager-789bdb4f85-rfnwn"
Jul 28 20:05:22.898216 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:22.898172    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zl9gp\" (UniqueName: \"kubernetes.io/projected/ec7bd9e6-90c1-468b-ba24-d274f070e2b9-kube-api-access-zl9gp\") pod \"lws-controller-manager-789bdb4f85-rfnwn\" (UID: \"ec7bd9e6-90c1-468b-ba24-d274f070e2b9\") " pod="openshift-lws-operator/lws-controller-manager-789bdb4f85-rfnwn"
Jul 28 20:05:22.898383 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:22.898260    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ec7bd9e6-90c1-468b-ba24-d274f070e2b9-cert\") pod \"lws-controller-manager-789bdb4f85-rfnwn\" (UID: \"ec7bd9e6-90c1-468b-ba24-d274f070e2b9\") " pod="openshift-lws-operator/lws-controller-manager-789bdb4f85-rfnwn"
Jul 28 20:05:22.898383 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:22.898284    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ec7bd9e6-90c1-468b-ba24-d274f070e2b9-manager-config\") pod \"lws-controller-manager-789bdb4f85-rfnwn\" (UID: \"ec7bd9e6-90c1-468b-ba24-d274f070e2b9\") " pod="openshift-lws-operator/lws-controller-manager-789bdb4f85-rfnwn"
Jul 28 20:05:22.898383 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:22.898304    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ec7bd9e6-90c1-468b-ba24-d274f070e2b9-metrics-cert\") pod \"lws-controller-manager-789bdb4f85-rfnwn\" (UID: \"ec7bd9e6-90c1-468b-ba24-d274f070e2b9\") " pod="openshift-lws-operator/lws-controller-manager-789bdb4f85-rfnwn"
Jul 28 20:05:22.898957 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:22.898924    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ec7bd9e6-90c1-468b-ba24-d274f070e2b9-manager-config\") pod \"lws-controller-manager-789bdb4f85-rfnwn\" (UID: \"ec7bd9e6-90c1-468b-ba24-d274f070e2b9\") " pod="openshift-lws-operator/lws-controller-manager-789bdb4f85-rfnwn"
Jul 28 20:05:22.900699 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:22.900677    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ec7bd9e6-90c1-468b-ba24-d274f070e2b9-cert\") pod \"lws-controller-manager-789bdb4f85-rfnwn\" (UID: \"ec7bd9e6-90c1-468b-ba24-d274f070e2b9\") " pod="openshift-lws-operator/lws-controller-manager-789bdb4f85-rfnwn"
Jul 28 20:05:22.900699 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:22.900693    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ec7bd9e6-90c1-468b-ba24-d274f070e2b9-metrics-cert\") pod \"lws-controller-manager-789bdb4f85-rfnwn\" (UID: \"ec7bd9e6-90c1-468b-ba24-d274f070e2b9\") " pod="openshift-lws-operator/lws-controller-manager-789bdb4f85-rfnwn"
Jul 28 20:05:22.913857 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:22.913826    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zl9gp\" (UniqueName: \"kubernetes.io/projected/ec7bd9e6-90c1-468b-ba24-d274f070e2b9-kube-api-access-zl9gp\") pod \"lws-controller-manager-789bdb4f85-rfnwn\" (UID: \"ec7bd9e6-90c1-468b-ba24-d274f070e2b9\") " pod="openshift-lws-operator/lws-controller-manager-789bdb4f85-rfnwn"
Jul 28 20:05:23.066039 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:23.065998    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-789bdb4f85-rfnwn"
Jul 28 20:05:23.198511 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:23.198452    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-789bdb4f85-rfnwn"]
Jul 28 20:05:23.201295 ip-10-0-139-54 kubenswrapper[2554]: W0728 20:05:23.201271    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec7bd9e6_90c1_468b_ba24_d274f070e2b9.slice/crio-643de8e54221f54ddcbad55db2955329920416216cc8e63e08ac3ce38067c1e1 WatchSource:0}: Error finding container 643de8e54221f54ddcbad55db2955329920416216cc8e63e08ac3ce38067c1e1: Status 404 returned error can't find the container with id 643de8e54221f54ddcbad55db2955329920416216cc8e63e08ac3ce38067c1e1
Jul 28 20:05:23.203165 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:23.203149    2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 20:05:23.550335 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:23.550300    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-789bdb4f85-rfnwn" event={"ID":"ec7bd9e6-90c1-468b-ba24-d274f070e2b9","Type":"ContainerStarted","Data":"643de8e54221f54ddcbad55db2955329920416216cc8e63e08ac3ce38067c1e1"}
Jul 28 20:05:23.562242 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:23.562206    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-685bc94585-57q9x"]
Jul 28 20:05:23.565311 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:23.565289    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-685bc94585-57q9x"
Jul 28 20:05:23.568693 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:23.568667    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 28 20:05:23.568818 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:23.568691    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 28 20:05:23.569353 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:23.569329    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 28 20:05:23.569499 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:23.569390    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 28 20:05:23.576941 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:23.576924    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-npzt2\""
Jul 28 20:05:23.603450 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:23.603418    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-685bc94585-57q9x"]
Jul 28 20:05:23.605063 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:23.605036    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwxwg\" (UniqueName: \"kubernetes.io/projected/86591ebe-d149-4205-9b93-0bfa8bc94e72-kube-api-access-zwxwg\") pod \"opendatahub-operator-controller-manager-685bc94585-57q9x\" (UID: \"86591ebe-d149-4205-9b93-0bfa8bc94e72\") " pod="opendatahub/opendatahub-operator-controller-manager-685bc94585-57q9x"
Jul 28 20:05:23.605204 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:23.605092    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/86591ebe-d149-4205-9b93-0bfa8bc94e72-webhook-cert\") pod \"opendatahub-operator-controller-manager-685bc94585-57q9x\" (UID: \"86591ebe-d149-4205-9b93-0bfa8bc94e72\") " pod="opendatahub/opendatahub-operator-controller-manager-685bc94585-57q9x"
Jul 28 20:05:23.605268 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:23.605200    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/86591ebe-d149-4205-9b93-0bfa8bc94e72-apiservice-cert\") pod \"opendatahub-operator-controller-manager-685bc94585-57q9x\" (UID: \"86591ebe-d149-4205-9b93-0bfa8bc94e72\") " pod="opendatahub/opendatahub-operator-controller-manager-685bc94585-57q9x"
Jul 28 20:05:23.605268 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:23.605251    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/86591ebe-d149-4205-9b93-0bfa8bc94e72-manifests\") pod \"opendatahub-operator-controller-manager-685bc94585-57q9x\" (UID: \"86591ebe-d149-4205-9b93-0bfa8bc94e72\") " pod="opendatahub/opendatahub-operator-controller-manager-685bc94585-57q9x"
Jul 28 20:05:23.605366 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:23.605297    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/86591ebe-d149-4205-9b93-0bfa8bc94e72-tmp\") pod \"opendatahub-operator-controller-manager-685bc94585-57q9x\" (UID: \"86591ebe-d149-4205-9b93-0bfa8bc94e72\") " pod="opendatahub/opendatahub-operator-controller-manager-685bc94585-57q9x"
Jul 28 20:05:23.706208 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:23.706166    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/86591ebe-d149-4205-9b93-0bfa8bc94e72-webhook-cert\") pod \"opendatahub-operator-controller-manager-685bc94585-57q9x\" (UID: \"86591ebe-d149-4205-9b93-0bfa8bc94e72\") " pod="opendatahub/opendatahub-operator-controller-manager-685bc94585-57q9x"
Jul 28 20:05:23.706419 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:23.706235    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/86591ebe-d149-4205-9b93-0bfa8bc94e72-apiservice-cert\") pod \"opendatahub-operator-controller-manager-685bc94585-57q9x\" (UID: \"86591ebe-d149-4205-9b93-0bfa8bc94e72\") " pod="opendatahub/opendatahub-operator-controller-manager-685bc94585-57q9x"
Jul 28 20:05:23.706419 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:23.706263    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/86591ebe-d149-4205-9b93-0bfa8bc94e72-manifests\") pod \"opendatahub-operator-controller-manager-685bc94585-57q9x\" (UID: \"86591ebe-d149-4205-9b93-0bfa8bc94e72\") " pod="opendatahub/opendatahub-operator-controller-manager-685bc94585-57q9x"
Jul 28 20:05:23.706419 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:23.706357    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/86591ebe-d149-4205-9b93-0bfa8bc94e72-tmp\") pod \"opendatahub-operator-controller-manager-685bc94585-57q9x\" (UID: \"86591ebe-d149-4205-9b93-0bfa8bc94e72\") " pod="opendatahub/opendatahub-operator-controller-manager-685bc94585-57q9x"
Jul 28 20:05:23.706599 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:23.706452    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwxwg\" (UniqueName: \"kubernetes.io/projected/86591ebe-d149-4205-9b93-0bfa8bc94e72-kube-api-access-zwxwg\") pod \"opendatahub-operator-controller-manager-685bc94585-57q9x\" (UID: \"86591ebe-d149-4205-9b93-0bfa8bc94e72\") " pod="opendatahub/opendatahub-operator-controller-manager-685bc94585-57q9x"
Jul 28 20:05:23.706702 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:23.706685    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/86591ebe-d149-4205-9b93-0bfa8bc94e72-manifests\") pod \"opendatahub-operator-controller-manager-685bc94585-57q9x\" (UID: \"86591ebe-d149-4205-9b93-0bfa8bc94e72\") " pod="opendatahub/opendatahub-operator-controller-manager-685bc94585-57q9x"
Jul 28 20:05:23.708552 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:23.708520    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/86591ebe-d149-4205-9b93-0bfa8bc94e72-tmp\") pod \"opendatahub-operator-controller-manager-685bc94585-57q9x\" (UID: \"86591ebe-d149-4205-9b93-0bfa8bc94e72\") " pod="opendatahub/opendatahub-operator-controller-manager-685bc94585-57q9x"
Jul 28 20:05:23.708652 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:23.708610    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/86591ebe-d149-4205-9b93-0bfa8bc94e72-webhook-cert\") pod \"opendatahub-operator-controller-manager-685bc94585-57q9x\" (UID: \"86591ebe-d149-4205-9b93-0bfa8bc94e72\") " pod="opendatahub/opendatahub-operator-controller-manager-685bc94585-57q9x"
Jul 28 20:05:23.708652 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:23.708635    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/86591ebe-d149-4205-9b93-0bfa8bc94e72-apiservice-cert\") pod \"opendatahub-operator-controller-manager-685bc94585-57q9x\" (UID: \"86591ebe-d149-4205-9b93-0bfa8bc94e72\") " pod="opendatahub/opendatahub-operator-controller-manager-685bc94585-57q9x"
Jul 28 20:05:23.720532 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:23.720505    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwxwg\" (UniqueName: \"kubernetes.io/projected/86591ebe-d149-4205-9b93-0bfa8bc94e72-kube-api-access-zwxwg\") pod \"opendatahub-operator-controller-manager-685bc94585-57q9x\" (UID: \"86591ebe-d149-4205-9b93-0bfa8bc94e72\") " pod="opendatahub/opendatahub-operator-controller-manager-685bc94585-57q9x"
Jul 28 20:05:23.875946 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:23.875847    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-685bc94585-57q9x"
Jul 28 20:05:24.016150 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:24.016123    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-685bc94585-57q9x"]
Jul 28 20:05:24.018929 ip-10-0-139-54 kubenswrapper[2554]: W0728 20:05:24.018895    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86591ebe_d149_4205_9b93_0bfa8bc94e72.slice/crio-d13318a4eeaf5c9daf39ad2f22af6f01bd971753dfc78027f439f5186f9cd0c2 WatchSource:0}: Error finding container d13318a4eeaf5c9daf39ad2f22af6f01bd971753dfc78027f439f5186f9cd0c2: Status 404 returned error can't find the container with id d13318a4eeaf5c9daf39ad2f22af6f01bd971753dfc78027f439f5186f9cd0c2
Jul 28 20:05:24.556809 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:24.556762    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-685bc94585-57q9x" event={"ID":"86591ebe-d149-4205-9b93-0bfa8bc94e72","Type":"ContainerStarted","Data":"d13318a4eeaf5c9daf39ad2f22af6f01bd971753dfc78027f439f5186f9cd0c2"}
Jul 28 20:05:28.578756 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:28.578714    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-685bc94585-57q9x" event={"ID":"86591ebe-d149-4205-9b93-0bfa8bc94e72","Type":"ContainerStarted","Data":"11c54e1dc7d20662914207960d0c22182eb9f6aebab514504bbfc8b38e51bcc0"}
Jul 28 20:05:28.580235 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:28.580208    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-789bdb4f85-rfnwn" event={"ID":"ec7bd9e6-90c1-468b-ba24-d274f070e2b9","Type":"ContainerStarted","Data":"2566e7a33102926b6a80c7f79998d78570322706fdd2f82053a74e43f386159c"}
Jul 28 20:05:28.580360 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:28.580341    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-789bdb4f85-rfnwn"
Jul 28 20:05:28.622094 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:28.622025    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-789bdb4f85-rfnwn" podStartSLOduration=1.975944547 podStartE2EDuration="6.622003046s" totalImagesPullingTime="4.646058499s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:05:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:05:28.620301377 +0000 UTC m=+391.403312815" watchObservedRunningTime="2026-07-28 20:05:28.622003046 +0000 UTC m=+391.405014488"
Jul 28 20:05:29.586066 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:29.586027    2554 generic.go:356] "Generic (PLEG): container finished" podID="86591ebe-d149-4205-9b93-0bfa8bc94e72" containerID="11c54e1dc7d20662914207960d0c22182eb9f6aebab514504bbfc8b38e51bcc0" exitCode=0
Jul 28 20:05:29.586470 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:29.586118    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-685bc94585-57q9x" event={"ID":"86591ebe-d149-4205-9b93-0bfa8bc94e72","Type":"ContainerDied","Data":"11c54e1dc7d20662914207960d0c22182eb9f6aebab514504bbfc8b38e51bcc0"}
Jul 28 20:05:30.591658 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:30.591622    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-685bc94585-57q9x" event={"ID":"86591ebe-d149-4205-9b93-0bfa8bc94e72","Type":"ContainerStarted","Data":"3f017fb397f55ff281e9065aeeb9a9fcf429c60c7aba914d0a98e4873fbaf425"}
Jul 28 20:05:30.592000 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:30.591746    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-685bc94585-57q9x"
Jul 28 20:05:30.613873 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:30.613821    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-685bc94585-57q9x" podStartSLOduration=2.119945507 podStartE2EDuration="7.613793631s" totalImagesPullingTime="4.402544844s" totalInitContainerRuntime="1.09130328s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:05:23 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:05:30.612454169 +0000 UTC m=+393.395465615" watchObservedRunningTime="2026-07-28 20:05:30.613793631 +0000 UTC m=+393.396805075"
Jul 28 20:05:39.588918 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:39.588886    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-789bdb4f85-rfnwn"
Jul 28 20:05:41.597684 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:41.597654    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-685bc94585-57q9x"
Jul 28 20:05:54.389723 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:54.389686    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6cdf7b948d-wtcxn"]
Jul 28 20:05:54.392841 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:54.392811    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6cdf7b948d-wtcxn"
Jul 28 20:05:54.396134 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:54.396108    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 28 20:05:54.396269 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:54.396110    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-s52fn\""
Jul 28 20:05:54.396269 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:54.396110    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 28 20:05:54.404893 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:54.404865    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6cdf7b948d-wtcxn"]
Jul 28 20:05:54.578091 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:54.578032    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57rf4\" (UniqueName: \"kubernetes.io/projected/44069363-b129-408c-90ce-a4c92b46d1b9-kube-api-access-57rf4\") pod \"kube-auth-proxy-6cdf7b948d-wtcxn\" (UID: \"44069363-b129-408c-90ce-a4c92b46d1b9\") " pod="openshift-ingress/kube-auth-proxy-6cdf7b948d-wtcxn"
Jul 28 20:05:54.578260 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:54.578148    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44069363-b129-408c-90ce-a4c92b46d1b9-tls-certs\") pod \"kube-auth-proxy-6cdf7b948d-wtcxn\" (UID: \"44069363-b129-408c-90ce-a4c92b46d1b9\") " pod="openshift-ingress/kube-auth-proxy-6cdf7b948d-wtcxn"
Jul 28 20:05:54.578260 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:54.578238    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/44069363-b129-408c-90ce-a4c92b46d1b9-tmp\") pod \"kube-auth-proxy-6cdf7b948d-wtcxn\" (UID: \"44069363-b129-408c-90ce-a4c92b46d1b9\") " pod="openshift-ingress/kube-auth-proxy-6cdf7b948d-wtcxn"
Jul 28 20:05:54.679365 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:54.679282    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/44069363-b129-408c-90ce-a4c92b46d1b9-tmp\") pod \"kube-auth-proxy-6cdf7b948d-wtcxn\" (UID: \"44069363-b129-408c-90ce-a4c92b46d1b9\") " pod="openshift-ingress/kube-auth-proxy-6cdf7b948d-wtcxn"
Jul 28 20:05:54.679365 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:54.679326    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57rf4\" (UniqueName: \"kubernetes.io/projected/44069363-b129-408c-90ce-a4c92b46d1b9-kube-api-access-57rf4\") pod \"kube-auth-proxy-6cdf7b948d-wtcxn\" (UID: \"44069363-b129-408c-90ce-a4c92b46d1b9\") " pod="openshift-ingress/kube-auth-proxy-6cdf7b948d-wtcxn"
Jul 28 20:05:54.679365 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:54.679357    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44069363-b129-408c-90ce-a4c92b46d1b9-tls-certs\") pod \"kube-auth-proxy-6cdf7b948d-wtcxn\" (UID: \"44069363-b129-408c-90ce-a4c92b46d1b9\") " pod="openshift-ingress/kube-auth-proxy-6cdf7b948d-wtcxn"
Jul 28 20:05:54.681724 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:54.681692    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/44069363-b129-408c-90ce-a4c92b46d1b9-tmp\") pod \"kube-auth-proxy-6cdf7b948d-wtcxn\" (UID: \"44069363-b129-408c-90ce-a4c92b46d1b9\") " pod="openshift-ingress/kube-auth-proxy-6cdf7b948d-wtcxn"
Jul 28 20:05:54.682028 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:54.682006    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44069363-b129-408c-90ce-a4c92b46d1b9-tls-certs\") pod \"kube-auth-proxy-6cdf7b948d-wtcxn\" (UID: \"44069363-b129-408c-90ce-a4c92b46d1b9\") " pod="openshift-ingress/kube-auth-proxy-6cdf7b948d-wtcxn"
Jul 28 20:05:54.693421 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:54.693397    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57rf4\" (UniqueName: \"kubernetes.io/projected/44069363-b129-408c-90ce-a4c92b46d1b9-kube-api-access-57rf4\") pod \"kube-auth-proxy-6cdf7b948d-wtcxn\" (UID: \"44069363-b129-408c-90ce-a4c92b46d1b9\") " pod="openshift-ingress/kube-auth-proxy-6cdf7b948d-wtcxn"
Jul 28 20:05:54.703189 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:54.703164    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6cdf7b948d-wtcxn"
Jul 28 20:05:54.839642 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:54.839615    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6cdf7b948d-wtcxn"]
Jul 28 20:05:54.841964 ip-10-0-139-54 kubenswrapper[2554]: W0728 20:05:54.841938    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44069363_b129_408c_90ce_a4c92b46d1b9.slice/crio-4d03ffc5a7a6c019c9f9b9109fc0003fbdd3d302ba6d3cbc74bc922ff1f90aa9 WatchSource:0}: Error finding container 4d03ffc5a7a6c019c9f9b9109fc0003fbdd3d302ba6d3cbc74bc922ff1f90aa9: Status 404 returned error can't find the container with id 4d03ffc5a7a6c019c9f9b9109fc0003fbdd3d302ba6d3cbc74bc922ff1f90aa9
Jul 28 20:05:55.697932 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:55.697893    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6cdf7b948d-wtcxn" event={"ID":"44069363-b129-408c-90ce-a4c92b46d1b9","Type":"ContainerStarted","Data":"4d03ffc5a7a6c019c9f9b9109fc0003fbdd3d302ba6d3cbc74bc922ff1f90aa9"}
Jul 28 20:05:57.709679 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:57.709594    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6cdf7b948d-wtcxn" event={"ID":"44069363-b129-408c-90ce-a4c92b46d1b9","Type":"ContainerStarted","Data":"f9fecbf66ae3a6339e88e6ea39392d33ddc33130171d52573e8f5457029cf78d"}
Jul 28 20:05:57.727638 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:05:57.727592    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6cdf7b948d-wtcxn" podStartSLOduration=1.164399733 podStartE2EDuration="3.727575937s" totalImagesPullingTime="2.563176204s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:05:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:05:57.726286523 +0000 UTC m=+420.509297955" watchObservedRunningTime="2026-07-28 20:05:57.727575937 +0000 UTC m=+420.510587418"
Jul 28 20:07:28.832636 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:07:28.832603    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-frxgw"]
Jul 28 20:07:28.835784 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:07:28.835759    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-frxgw"
Jul 28 20:07:28.838595 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:07:28.838574    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 28 20:07:28.839893 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:07:28.839869    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-mdmdb\""
Jul 28 20:07:28.840006 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:07:28.839895    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 28 20:07:28.840006 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:07:28.839869    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 28 20:07:28.844240 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:07:28.844214    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-frxgw"]
Jul 28 20:07:28.915136 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:07:28.915084    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6njc\" (UniqueName: \"kubernetes.io/projected/30697c07-d9ef-4436-93b0-e7f140368a1d-kube-api-access-f6njc\") pod \"dns-operator-controller-manager-864dd78c4c-frxgw\" (UID: \"30697c07-d9ef-4436-93b0-e7f140368a1d\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-frxgw"
Jul 28 20:07:29.015847 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:07:29.015810    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6njc\" (UniqueName: \"kubernetes.io/projected/30697c07-d9ef-4436-93b0-e7f140368a1d-kube-api-access-f6njc\") pod \"dns-operator-controller-manager-864dd78c4c-frxgw\" (UID: \"30697c07-d9ef-4436-93b0-e7f140368a1d\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-frxgw"
Jul 28 20:07:29.024843 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:07:29.024820    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6njc\" (UniqueName: \"kubernetes.io/projected/30697c07-d9ef-4436-93b0-e7f140368a1d-kube-api-access-f6njc\") pod \"dns-operator-controller-manager-864dd78c4c-frxgw\" (UID: \"30697c07-d9ef-4436-93b0-e7f140368a1d\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-frxgw"
Jul 28 20:07:29.146234 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:07:29.146149    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-frxgw"
Jul 28 20:07:29.276855 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:07:29.276826    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-frxgw"]
Jul 28 20:07:29.279650 ip-10-0-139-54 kubenswrapper[2554]: W0728 20:07:29.279626    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30697c07_d9ef_4436_93b0_e7f140368a1d.slice/crio-c5409c703d119256a015ea8f309d2229e3d9155cb30b71d0d96a3d53b9a55028 WatchSource:0}: Error finding container c5409c703d119256a015ea8f309d2229e3d9155cb30b71d0d96a3d53b9a55028: Status 404 returned error can't find the container with id c5409c703d119256a015ea8f309d2229e3d9155cb30b71d0d96a3d53b9a55028
Jul 28 20:07:30.095590 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:07:30.095555    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-frxgw" event={"ID":"30697c07-d9ef-4436-93b0-e7f140368a1d","Type":"ContainerStarted","Data":"c5409c703d119256a015ea8f309d2229e3d9155cb30b71d0d96a3d53b9a55028"}
Jul 28 20:07:32.105152 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:07:32.105116    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-frxgw" event={"ID":"30697c07-d9ef-4436-93b0-e7f140368a1d","Type":"ContainerStarted","Data":"a97ed158d29183221013523382987259c52c8f39bb718a10d867222cd820381e"}
Jul 28 20:07:32.105514 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:07:32.105243    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-frxgw"
Jul 28 20:07:32.125855 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:07:32.125811    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-frxgw" podStartSLOduration=1.771331044 podStartE2EDuration="4.125796441s" totalImagesPullingTime="2.354465397s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:07:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:07:32.125335535 +0000 UTC m=+514.908346978" watchObservedRunningTime="2026-07-28 20:07:32.125796441 +0000 UTC m=+514.908807883"
Jul 28 20:07:34.672929 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:07:34.672888    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-gl8p6"]
Jul 28 20:07:34.676092 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:07:34.676059    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gl8p6"
Jul 28 20:07:34.678811 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:07:34.678789    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 28 20:07:34.678941 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:07:34.678828    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 28 20:07:34.678941 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:07:34.678909    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-dspjd\""
Jul 28 20:07:34.683931 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:07:34.683908    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-gl8p6"]
Jul 28 20:07:34.763357 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:07:34.763311    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7b5bc3d2-d2f5-4944-bcb0-f45ab40c4dc0-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-gl8p6\" (UID: \"7b5bc3d2-d2f5-4944-bcb0-f45ab40c4dc0\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gl8p6"
Jul 28 20:07:34.763357 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:07:34.763359    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/7b5bc3d2-d2f5-4944-bcb0-f45ab40c4dc0-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-gl8p6\" (UID: \"7b5bc3d2-d2f5-4944-bcb0-f45ab40c4dc0\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gl8p6"
Jul 28 20:07:34.763575 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:07:34.763430    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpv6t\" (UniqueName: \"kubernetes.io/projected/7b5bc3d2-d2f5-4944-bcb0-f45ab40c4dc0-kube-api-access-cpv6t\") pod \"kuadrant-console-plugin-66b5774985-gl8p6\" (UID: \"7b5bc3d2-d2f5-4944-bcb0-f45ab40c4dc0\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gl8p6"
Jul 28 20:07:34.864861 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:07:34.864822    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7b5bc3d2-d2f5-4944-bcb0-f45ab40c4dc0-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-gl8p6\" (UID: \"7b5bc3d2-d2f5-4944-bcb0-f45ab40c4dc0\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gl8p6"
Jul 28 20:07:34.864861 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:07:34.864858    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/7b5bc3d2-d2f5-4944-bcb0-f45ab40c4dc0-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-gl8p6\" (UID: \"7b5bc3d2-d2f5-4944-bcb0-f45ab40c4dc0\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gl8p6"
Jul 28 20:07:34.865177 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:07:34.864957    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpv6t\" (UniqueName: \"kubernetes.io/projected/7b5bc3d2-d2f5-4944-bcb0-f45ab40c4dc0-kube-api-access-cpv6t\") pod \"kuadrant-console-plugin-66b5774985-gl8p6\" (UID: \"7b5bc3d2-d2f5-4944-bcb0-f45ab40c4dc0\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gl8p6"
Jul 28 20:07:34.865177 ip-10-0-139-54 kubenswrapper[2554]: E0728 20:07:34.865097    2554 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 28 20:07:34.865177 ip-10-0-139-54 kubenswrapper[2554]: E0728 20:07:34.865170    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b5bc3d2-d2f5-4944-bcb0-f45ab40c4dc0-plugin-serving-cert podName:7b5bc3d2-d2f5-4944-bcb0-f45ab40c4dc0 nodeName:}" failed. No retries permitted until 2026-07-28 20:07:35.365151854 +0000 UTC m=+518.148163274 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/7b5bc3d2-d2f5-4944-bcb0-f45ab40c4dc0-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-gl8p6" (UID: "7b5bc3d2-d2f5-4944-bcb0-f45ab40c4dc0") : secret "plugin-serving-cert" not found
Jul 28 20:07:34.865552 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:07:34.865528    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7b5bc3d2-d2f5-4944-bcb0-f45ab40c4dc0-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-gl8p6\" (UID: \"7b5bc3d2-d2f5-4944-bcb0-f45ab40c4dc0\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gl8p6"
Jul 28 20:07:34.877775 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:07:34.877744    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpv6t\" (UniqueName: \"kubernetes.io/projected/7b5bc3d2-d2f5-4944-bcb0-f45ab40c4dc0-kube-api-access-cpv6t\") pod \"kuadrant-console-plugin-66b5774985-gl8p6\" (UID: \"7b5bc3d2-d2f5-4944-bcb0-f45ab40c4dc0\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gl8p6"
Jul 28 20:07:35.369387 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:07:35.369322    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/7b5bc3d2-d2f5-4944-bcb0-f45ab40c4dc0-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-gl8p6\" (UID: \"7b5bc3d2-d2f5-4944-bcb0-f45ab40c4dc0\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gl8p6"
Jul 28 20:07:35.371725 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:07:35.371691    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/7b5bc3d2-d2f5-4944-bcb0-f45ab40c4dc0-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-gl8p6\" (UID: \"7b5bc3d2-d2f5-4944-bcb0-f45ab40c4dc0\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gl8p6"
Jul 28 20:07:35.594309 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:07:35.594272    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gl8p6"
Jul 28 20:07:35.724507 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:07:35.724482    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-gl8p6"]
Jul 28 20:07:35.727330 ip-10-0-139-54 kubenswrapper[2554]: W0728 20:07:35.727290    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7b5bc3d2_d2f5_4944_bcb0_f45ab40c4dc0.slice/crio-dc8cd3263bcaeec18fafdec310966fcc9b07b73e1d0d4e6bdd27ac7684af8612 WatchSource:0}: Error finding container dc8cd3263bcaeec18fafdec310966fcc9b07b73e1d0d4e6bdd27ac7684af8612: Status 404 returned error can't find the container with id dc8cd3263bcaeec18fafdec310966fcc9b07b73e1d0d4e6bdd27ac7684af8612
Jul 28 20:07:36.123732 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:07:36.123689    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gl8p6" event={"ID":"7b5bc3d2-d2f5-4944-bcb0-f45ab40c4dc0","Type":"ContainerStarted","Data":"dc8cd3263bcaeec18fafdec310966fcc9b07b73e1d0d4e6bdd27ac7684af8612"}
Jul 28 20:07:43.111879 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:07:43.111846    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-frxgw"
Jul 28 20:07:45.283870 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:07:45.283833    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vp6pt"]
Jul 28 20:07:45.294063 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:07:45.294031    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vp6pt"
Jul 28 20:07:45.297284 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:07:45.297257    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-4sg66\""
Jul 28 20:07:45.300999 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:07:45.300962    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vp6pt"]
Jul 28 20:07:45.364432 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:07:45.364382    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfchr\" (UniqueName: \"kubernetes.io/projected/cbedc35d-405b-421c-ba08-f7051525b560-kube-api-access-bfchr\") pod \"limitador-operator-controller-manager-859d7b78bd-vp6pt\" (UID: \"cbedc35d-405b-421c-ba08-f7051525b560\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vp6pt"
Jul 28 20:07:45.465097 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:07:45.465046    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfchr\" (UniqueName: \"kubernetes.io/projected/cbedc35d-405b-421c-ba08-f7051525b560-kube-api-access-bfchr\") pod \"limitador-operator-controller-manager-859d7b78bd-vp6pt\" (UID: \"cbedc35d-405b-421c-ba08-f7051525b560\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vp6pt"
Jul 28 20:07:45.476244 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:07:45.476207    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfchr\" (UniqueName: \"kubernetes.io/projected/cbedc35d-405b-421c-ba08-f7051525b560-kube-api-access-bfchr\") pod \"limitador-operator-controller-manager-859d7b78bd-vp6pt\" (UID: \"cbedc35d-405b-421c-ba08-f7051525b560\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vp6pt"
Jul 28 20:07:45.609020 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:07:45.608972    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vp6pt"
Jul 28 20:07:58.543385 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:07:58.543355    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vp6pt"]
Jul 28 20:07:58.545717 ip-10-0-139-54 kubenswrapper[2554]: W0728 20:07:58.545685    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbedc35d_405b_421c_ba08_f7051525b560.slice/crio-6f95da9318dabd7f11ba8b1f5ed36ed6c584b01b7a640013be570ac2d44ad923 WatchSource:0}: Error finding container 6f95da9318dabd7f11ba8b1f5ed36ed6c584b01b7a640013be570ac2d44ad923: Status 404 returned error can't find the container with id 6f95da9318dabd7f11ba8b1f5ed36ed6c584b01b7a640013be570ac2d44ad923
Jul 28 20:07:59.237309 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:07:59.237275    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gl8p6" event={"ID":"7b5bc3d2-d2f5-4944-bcb0-f45ab40c4dc0","Type":"ContainerStarted","Data":"2f4b7b1f16f5b29eb4aab6f8ff364c968f4f6c3eb297980d045c52daadb270bc"}
Jul 28 20:07:59.238661 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:07:59.238619    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vp6pt" event={"ID":"cbedc35d-405b-421c-ba08-f7051525b560","Type":"ContainerStarted","Data":"6f95da9318dabd7f11ba8b1f5ed36ed6c584b01b7a640013be570ac2d44ad923"}
Jul 28 20:07:59.258629 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:07:59.258550    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gl8p6" podStartSLOduration=2.489766537 podStartE2EDuration="25.258529526s" totalImagesPullingTime="22.768762989s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:07:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:07:59.258430684 +0000 UTC m=+542.041442125" watchObservedRunningTime="2026-07-28 20:07:59.258529526 +0000 UTC m=+542.041540968"
Jul 28 20:08:01.248644 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:08:01.248607    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vp6pt" event={"ID":"cbedc35d-405b-421c-ba08-f7051525b560","Type":"ContainerStarted","Data":"b0eca06f5dedfb1d26b339e13b2f84d3a0381d5e30a9904a513f759755d55b1d"}
Jul 28 20:08:01.249180 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:08:01.248752    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vp6pt"
Jul 28 20:08:01.268190 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:08:01.268136    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vp6pt" podStartSLOduration=14.623395189 podStartE2EDuration="16.268115122s" totalImagesPullingTime="1.644719933s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:07:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:08:01.267545172 +0000 UTC m=+544.050556646" watchObservedRunningTime="2026-07-28 20:08:01.268115122 +0000 UTC m=+544.051126563"
Jul 28 20:08:12.255242 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:08:12.255212    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vp6pt"
Jul 28 20:08:48.908714 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:08:48.908678    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-spcc9"]
Jul 28 20:08:48.960549 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:08:48.960511    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-spcc9"]
Jul 28 20:08:48.960711 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:08:48.960616    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-spcc9"
Jul 28 20:08:48.963645 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:08:48.963622    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-hmxqt\""
Jul 28 20:08:49.022744 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:08:49.022708    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqv7m\" (UniqueName: \"kubernetes.io/projected/1b9c8ff9-46e6-436f-a7b9-cb5bb101752a-kube-api-access-lqv7m\") pod \"authorino-75d556bd77-spcc9\" (UID: \"1b9c8ff9-46e6-436f-a7b9-cb5bb101752a\") " pod="kuadrant-system/authorino-75d556bd77-spcc9"
Jul 28 20:08:49.123973 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:08:49.123923    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqv7m\" (UniqueName: \"kubernetes.io/projected/1b9c8ff9-46e6-436f-a7b9-cb5bb101752a-kube-api-access-lqv7m\") pod \"authorino-75d556bd77-spcc9\" (UID: \"1b9c8ff9-46e6-436f-a7b9-cb5bb101752a\") " pod="kuadrant-system/authorino-75d556bd77-spcc9"
Jul 28 20:08:49.132891 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:08:49.132857    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqv7m\" (UniqueName: \"kubernetes.io/projected/1b9c8ff9-46e6-436f-a7b9-cb5bb101752a-kube-api-access-lqv7m\") pod \"authorino-75d556bd77-spcc9\" (UID: \"1b9c8ff9-46e6-436f-a7b9-cb5bb101752a\") " pod="kuadrant-system/authorino-75d556bd77-spcc9"
Jul 28 20:08:49.269684 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:08:49.269599    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-spcc9"
Jul 28 20:08:49.400891 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:08:49.400846    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-spcc9"]
Jul 28 20:08:49.405132 ip-10-0-139-54 kubenswrapper[2554]: W0728 20:08:49.405094    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1b9c8ff9_46e6_436f_a7b9_cb5bb101752a.slice/crio-ed8560e0b7f41490e952465d2268e5ed0f4ee86a021af6280a0f8644481a3916 WatchSource:0}: Error finding container ed8560e0b7f41490e952465d2268e5ed0f4ee86a021af6280a0f8644481a3916: Status 404 returned error can't find the container with id ed8560e0b7f41490e952465d2268e5ed0f4ee86a021af6280a0f8644481a3916
Jul 28 20:08:49.455686 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:08:49.455650    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-spcc9" event={"ID":"1b9c8ff9-46e6-436f-a7b9-cb5bb101752a","Type":"ContainerStarted","Data":"ed8560e0b7f41490e952465d2268e5ed0f4ee86a021af6280a0f8644481a3916"}
Jul 28 20:08:52.473326 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:08:52.473294    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-spcc9" event={"ID":"1b9c8ff9-46e6-436f-a7b9-cb5bb101752a","Type":"ContainerStarted","Data":"51a96eae27a1ac39c8a524bb3ea750c148bd2542613d00d3896d08181cc5f181"}
Jul 28 20:08:52.488788 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:08:52.488700    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-spcc9" podStartSLOduration=1.6554354770000002 podStartE2EDuration="4.488685592s" totalImagesPullingTime="2.833250115s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:08:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:08:52.487444636 +0000 UTC m=+595.270456080" watchObservedRunningTime="2026-07-28 20:08:52.488685592 +0000 UTC m=+595.271697034"
Jul 28 20:08:57.709054 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:08:57.709024    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-7vcr8_dde4c1a8-8229-4419-87b6-d8e39a0827c4/console-operator/1.log"
Jul 28 20:08:57.711226 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:08:57.711205    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-7vcr8_dde4c1a8-8229-4419-87b6-d8e39a0827c4/console-operator/1.log"
Jul 28 20:09:25.115895 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:09:25.115810    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-spcc9"]
Jul 28 20:09:25.116607 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:09:25.116039    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-spcc9" podUID="1b9c8ff9-46e6-436f-a7b9-cb5bb101752a" containerName="authorino" containerID="cri-o://51a96eae27a1ac39c8a524bb3ea750c148bd2542613d00d3896d08181cc5f181" gracePeriod=30
Jul 28 20:09:25.356136 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:09:25.356113    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-spcc9"
Jul 28 20:09:25.422923 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:09:25.422826    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lqv7m\" (UniqueName: \"kubernetes.io/projected/1b9c8ff9-46e6-436f-a7b9-cb5bb101752a-kube-api-access-lqv7m\") pod \"1b9c8ff9-46e6-436f-a7b9-cb5bb101752a\" (UID: \"1b9c8ff9-46e6-436f-a7b9-cb5bb101752a\") "
Jul 28 20:09:25.425013 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:09:25.424988    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b9c8ff9-46e6-436f-a7b9-cb5bb101752a-kube-api-access-lqv7m" (OuterVolumeSpecName: "kube-api-access-lqv7m") pod "1b9c8ff9-46e6-436f-a7b9-cb5bb101752a" (UID: "1b9c8ff9-46e6-436f-a7b9-cb5bb101752a"). InnerVolumeSpecName "kube-api-access-lqv7m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:09:25.523802 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:09:25.523758    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lqv7m\" (UniqueName: \"kubernetes.io/projected/1b9c8ff9-46e6-436f-a7b9-cb5bb101752a-kube-api-access-lqv7m\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:09:25.618402 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:09:25.618372    2554 generic.go:356] "Generic (PLEG): container finished" podID="1b9c8ff9-46e6-436f-a7b9-cb5bb101752a" containerID="51a96eae27a1ac39c8a524bb3ea750c148bd2542613d00d3896d08181cc5f181" exitCode=0
Jul 28 20:09:25.618569 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:09:25.618415    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-spcc9"
Jul 28 20:09:25.618569 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:09:25.618454    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-spcc9" event={"ID":"1b9c8ff9-46e6-436f-a7b9-cb5bb101752a","Type":"ContainerDied","Data":"51a96eae27a1ac39c8a524bb3ea750c148bd2542613d00d3896d08181cc5f181"}
Jul 28 20:09:25.618569 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:09:25.618500    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-spcc9" event={"ID":"1b9c8ff9-46e6-436f-a7b9-cb5bb101752a","Type":"ContainerDied","Data":"ed8560e0b7f41490e952465d2268e5ed0f4ee86a021af6280a0f8644481a3916"}
Jul 28 20:09:25.618569 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:09:25.618524    2554 scope.go:117] "RemoveContainer" containerID="51a96eae27a1ac39c8a524bb3ea750c148bd2542613d00d3896d08181cc5f181"
Jul 28 20:09:25.627365 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:09:25.627347    2554 scope.go:117] "RemoveContainer" containerID="51a96eae27a1ac39c8a524bb3ea750c148bd2542613d00d3896d08181cc5f181"
Jul 28 20:09:25.627620 ip-10-0-139-54 kubenswrapper[2554]: E0728 20:09:25.627601    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51a96eae27a1ac39c8a524bb3ea750c148bd2542613d00d3896d08181cc5f181\": container with ID starting with 51a96eae27a1ac39c8a524bb3ea750c148bd2542613d00d3896d08181cc5f181 not found: ID does not exist" containerID="51a96eae27a1ac39c8a524bb3ea750c148bd2542613d00d3896d08181cc5f181"
Jul 28 20:09:25.627678 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:09:25.627630    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51a96eae27a1ac39c8a524bb3ea750c148bd2542613d00d3896d08181cc5f181"} err="failed to get container status \"51a96eae27a1ac39c8a524bb3ea750c148bd2542613d00d3896d08181cc5f181\": rpc error: code = NotFound desc = could not find container \"51a96eae27a1ac39c8a524bb3ea750c148bd2542613d00d3896d08181cc5f181\": container with ID starting with 51a96eae27a1ac39c8a524bb3ea750c148bd2542613d00d3896d08181cc5f181 not found: ID does not exist"
Jul 28 20:09:25.645011 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:09:25.644984    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-spcc9"]
Jul 28 20:09:25.650172 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:09:25.650149    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-spcc9"]
Jul 28 20:09:25.777971 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:09:25.777900    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b9c8ff9-46e6-436f-a7b9-cb5bb101752a" path="/var/lib/kubelet/pods/1b9c8ff9-46e6-436f-a7b9-cb5bb101752a/volumes"
Jul 28 20:10:57.486750 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:10:57.486661    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-zx8p7"]
Jul 28 20:10:57.487293 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:10:57.487146    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b9c8ff9-46e6-436f-a7b9-cb5bb101752a" containerName="authorino"
Jul 28 20:10:57.487293 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:10:57.487166    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b9c8ff9-46e6-436f-a7b9-cb5bb101752a" containerName="authorino"
Jul 28 20:10:57.487293 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:10:57.487278    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b9c8ff9-46e6-436f-a7b9-cb5bb101752a" containerName="authorino"
Jul 28 20:10:57.490277 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:10:57.490255    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zx8p7"
Jul 28 20:10:57.494284 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:10:57.494264    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 28 20:10:57.494284 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:10:57.494276    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 28 20:10:57.494456 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:10:57.494296    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 28 20:10:57.494456 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:10:57.494276    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-rnpl5\""
Jul 28 20:10:57.506566 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:10:57.506540    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-zx8p7"]
Jul 28 20:10:57.527552 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:10:57.527515    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09cc7986-c2a0-49d2-8214-0abfd81d6b8a-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zx8p7\" (UID: \"09cc7986-c2a0-49d2-8214-0abfd81d6b8a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zx8p7"
Jul 28 20:10:57.527715 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:10:57.527559    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09cc7986-c2a0-49d2-8214-0abfd81d6b8a-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zx8p7\" (UID: \"09cc7986-c2a0-49d2-8214-0abfd81d6b8a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zx8p7"
Jul 28 20:10:57.527715 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:10:57.527586    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09cc7986-c2a0-49d2-8214-0abfd81d6b8a-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zx8p7\" (UID: \"09cc7986-c2a0-49d2-8214-0abfd81d6b8a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zx8p7"
Jul 28 20:10:57.527715 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:10:57.527620    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09cc7986-c2a0-49d2-8214-0abfd81d6b8a-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zx8p7\" (UID: \"09cc7986-c2a0-49d2-8214-0abfd81d6b8a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zx8p7"
Jul 28 20:10:57.527715 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:10:57.527685    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09cc7986-c2a0-49d2-8214-0abfd81d6b8a-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zx8p7\" (UID: \"09cc7986-c2a0-49d2-8214-0abfd81d6b8a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zx8p7"
Jul 28 20:10:57.527872 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:10:57.527731    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9jdw\" (UniqueName: \"kubernetes.io/projected/09cc7986-c2a0-49d2-8214-0abfd81d6b8a-kube-api-access-v9jdw\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zx8p7\" (UID: \"09cc7986-c2a0-49d2-8214-0abfd81d6b8a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zx8p7"
Jul 28 20:10:57.628676 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:10:57.628634    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09cc7986-c2a0-49d2-8214-0abfd81d6b8a-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zx8p7\" (UID: \"09cc7986-c2a0-49d2-8214-0abfd81d6b8a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zx8p7"
Jul 28 20:10:57.628853 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:10:57.628700    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9jdw\" (UniqueName: \"kubernetes.io/projected/09cc7986-c2a0-49d2-8214-0abfd81d6b8a-kube-api-access-v9jdw\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zx8p7\" (UID: \"09cc7986-c2a0-49d2-8214-0abfd81d6b8a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zx8p7"
Jul 28 20:10:57.628853 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:10:57.628777    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09cc7986-c2a0-49d2-8214-0abfd81d6b8a-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zx8p7\" (UID: \"09cc7986-c2a0-49d2-8214-0abfd81d6b8a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zx8p7"
Jul 28 20:10:57.628853 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:10:57.628806    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09cc7986-c2a0-49d2-8214-0abfd81d6b8a-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zx8p7\" (UID: \"09cc7986-c2a0-49d2-8214-0abfd81d6b8a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zx8p7"
Jul 28 20:10:57.629007 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:10:57.628972    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09cc7986-c2a0-49d2-8214-0abfd81d6b8a-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zx8p7\" (UID: \"09cc7986-c2a0-49d2-8214-0abfd81d6b8a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zx8p7"
Jul 28 20:10:57.629055 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:10:57.629020    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09cc7986-c2a0-49d2-8214-0abfd81d6b8a-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zx8p7\" (UID: \"09cc7986-c2a0-49d2-8214-0abfd81d6b8a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zx8p7"
Jul 28 20:10:57.629159 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:10:57.629138    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09cc7986-c2a0-49d2-8214-0abfd81d6b8a-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zx8p7\" (UID: \"09cc7986-c2a0-49d2-8214-0abfd81d6b8a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zx8p7"
Jul 28 20:10:57.629159 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:10:57.629149    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09cc7986-c2a0-49d2-8214-0abfd81d6b8a-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zx8p7\" (UID: \"09cc7986-c2a0-49d2-8214-0abfd81d6b8a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zx8p7"
Jul 28 20:10:57.629335 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:10:57.629311    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09cc7986-c2a0-49d2-8214-0abfd81d6b8a-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zx8p7\" (UID: \"09cc7986-c2a0-49d2-8214-0abfd81d6b8a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zx8p7"
Jul 28 20:10:57.631044 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:10:57.631023    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09cc7986-c2a0-49d2-8214-0abfd81d6b8a-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zx8p7\" (UID: \"09cc7986-c2a0-49d2-8214-0abfd81d6b8a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zx8p7"
Jul 28 20:10:57.631311 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:10:57.631294    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09cc7986-c2a0-49d2-8214-0abfd81d6b8a-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zx8p7\" (UID: \"09cc7986-c2a0-49d2-8214-0abfd81d6b8a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zx8p7"
Jul 28 20:10:57.636959 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:10:57.636937    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9jdw\" (UniqueName: \"kubernetes.io/projected/09cc7986-c2a0-49d2-8214-0abfd81d6b8a-kube-api-access-v9jdw\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zx8p7\" (UID: \"09cc7986-c2a0-49d2-8214-0abfd81d6b8a\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zx8p7"
Jul 28 20:10:57.803738 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:10:57.803709    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-rnpl5\""
Jul 28 20:10:57.811916 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:10:57.811884    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zx8p7"
Jul 28 20:10:57.948182 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:10:57.948154    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-zx8p7"]
Jul 28 20:10:57.950099 ip-10-0-139-54 kubenswrapper[2554]: W0728 20:10:57.950058    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09cc7986_c2a0_49d2_8214_0abfd81d6b8a.slice/crio-7da7edb4537e8be11ec7ffbf8a88482ddc205a3760596fae5f7fa0fb45744cc6 WatchSource:0}: Error finding container 7da7edb4537e8be11ec7ffbf8a88482ddc205a3760596fae5f7fa0fb45744cc6: Status 404 returned error can't find the container with id 7da7edb4537e8be11ec7ffbf8a88482ddc205a3760596fae5f7fa0fb45744cc6
Jul 28 20:10:57.951571 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:10:57.951549    2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 20:10:58.019433 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:10:58.019397    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zx8p7" event={"ID":"09cc7986-c2a0-49d2-8214-0abfd81d6b8a","Type":"ContainerStarted","Data":"7da7edb4537e8be11ec7ffbf8a88482ddc205a3760596fae5f7fa0fb45744cc6"}
Jul 28 20:10:59.025356 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:10:59.025317    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zx8p7" event={"ID":"09cc7986-c2a0-49d2-8214-0abfd81d6b8a","Type":"ContainerStarted","Data":"6ff9a02a56eeca867a262f6412b6f9828d0c4a5944eb9744c07b5b9b870ada2d"}
Jul 28 20:10:59.042897 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:10:59.042850    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zx8p7" podStartSLOduration=1.034104221 podStartE2EDuration="2.042831367s" totalImagesPullingTime="1.008727146s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:10:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:10:59.042450137 +0000 UTC m=+721.825461582" watchObservedRunningTime="2026-07-28 20:10:59.042831367 +0000 UTC m=+721.825842810"
Jul 28 20:11:05.277989 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:05.277954    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-49sbr"]
Jul 28 20:11:05.281526 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:05.281503    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-49sbr"
Jul 28 20:11:05.284433 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:05.284413    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 28 20:11:05.295307 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:05.295283    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-49sbr"]
Jul 28 20:11:05.396925 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:05.396889    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8131278e-1e59-4a56-83a2-ed9a0b882c98-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-49sbr\" (UID: \"8131278e-1e59-4a56-83a2-ed9a0b882c98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-49sbr"
Jul 28 20:11:05.396925 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:05.396936    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8131278e-1e59-4a56-83a2-ed9a0b882c98-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-49sbr\" (UID: \"8131278e-1e59-4a56-83a2-ed9a0b882c98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-49sbr"
Jul 28 20:11:05.397226 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:05.396953    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8131278e-1e59-4a56-83a2-ed9a0b882c98-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-49sbr\" (UID: \"8131278e-1e59-4a56-83a2-ed9a0b882c98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-49sbr"
Jul 28 20:11:05.397226 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:05.396973    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8131278e-1e59-4a56-83a2-ed9a0b882c98-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-49sbr\" (UID: \"8131278e-1e59-4a56-83a2-ed9a0b882c98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-49sbr"
Jul 28 20:11:05.397226 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:05.397058    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-555pb\" (UniqueName: \"kubernetes.io/projected/8131278e-1e59-4a56-83a2-ed9a0b882c98-kube-api-access-555pb\") pod \"e2e-trlp-test-simulated-kserve-598466548d-49sbr\" (UID: \"8131278e-1e59-4a56-83a2-ed9a0b882c98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-49sbr"
Jul 28 20:11:05.397226 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:05.397171    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8131278e-1e59-4a56-83a2-ed9a0b882c98-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-49sbr\" (UID: \"8131278e-1e59-4a56-83a2-ed9a0b882c98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-49sbr"
Jul 28 20:11:05.498158 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:05.498118    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8131278e-1e59-4a56-83a2-ed9a0b882c98-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-49sbr\" (UID: \"8131278e-1e59-4a56-83a2-ed9a0b882c98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-49sbr"
Jul 28 20:11:05.498321 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:05.498172    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8131278e-1e59-4a56-83a2-ed9a0b882c98-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-49sbr\" (UID: \"8131278e-1e59-4a56-83a2-ed9a0b882c98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-49sbr"
Jul 28 20:11:05.498321 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:05.498199    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8131278e-1e59-4a56-83a2-ed9a0b882c98-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-49sbr\" (UID: \"8131278e-1e59-4a56-83a2-ed9a0b882c98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-49sbr"
Jul 28 20:11:05.498321 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:05.498226    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8131278e-1e59-4a56-83a2-ed9a0b882c98-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-49sbr\" (UID: \"8131278e-1e59-4a56-83a2-ed9a0b882c98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-49sbr"
Jul 28 20:11:05.498321 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:05.498258    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-555pb\" (UniqueName: \"kubernetes.io/projected/8131278e-1e59-4a56-83a2-ed9a0b882c98-kube-api-access-555pb\") pod \"e2e-trlp-test-simulated-kserve-598466548d-49sbr\" (UID: \"8131278e-1e59-4a56-83a2-ed9a0b882c98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-49sbr"
Jul 28 20:11:05.498321 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:05.498318    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8131278e-1e59-4a56-83a2-ed9a0b882c98-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-49sbr\" (UID: \"8131278e-1e59-4a56-83a2-ed9a0b882c98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-49sbr"
Jul 28 20:11:05.498623 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:05.498590    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8131278e-1e59-4a56-83a2-ed9a0b882c98-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-49sbr\" (UID: \"8131278e-1e59-4a56-83a2-ed9a0b882c98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-49sbr"
Jul 28 20:11:05.498706 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:05.498640    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8131278e-1e59-4a56-83a2-ed9a0b882c98-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-49sbr\" (UID: \"8131278e-1e59-4a56-83a2-ed9a0b882c98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-49sbr"
Jul 28 20:11:05.498761 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:05.498699    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8131278e-1e59-4a56-83a2-ed9a0b882c98-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-49sbr\" (UID: \"8131278e-1e59-4a56-83a2-ed9a0b882c98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-49sbr"
Jul 28 20:11:05.500522 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:05.500499    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8131278e-1e59-4a56-83a2-ed9a0b882c98-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-49sbr\" (UID: \"8131278e-1e59-4a56-83a2-ed9a0b882c98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-49sbr"
Jul 28 20:11:05.500673 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:05.500654    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8131278e-1e59-4a56-83a2-ed9a0b882c98-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-49sbr\" (UID: \"8131278e-1e59-4a56-83a2-ed9a0b882c98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-49sbr"
Jul 28 20:11:05.506238 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:05.506218    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-555pb\" (UniqueName: \"kubernetes.io/projected/8131278e-1e59-4a56-83a2-ed9a0b882c98-kube-api-access-555pb\") pod \"e2e-trlp-test-simulated-kserve-598466548d-49sbr\" (UID: \"8131278e-1e59-4a56-83a2-ed9a0b882c98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-49sbr"
Jul 28 20:11:05.591263 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:05.591231    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-49sbr"
Jul 28 20:11:05.719596 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:05.719514    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-49sbr"]
Jul 28 20:11:05.722421 ip-10-0-139-54 kubenswrapper[2554]: W0728 20:11:05.722394    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8131278e_1e59_4a56_83a2_ed9a0b882c98.slice/crio-15afd821174604e88d650c6765f46aee0cc1e9cfb0b9fa8894dbd6739bd5a6e7 WatchSource:0}: Error finding container 15afd821174604e88d650c6765f46aee0cc1e9cfb0b9fa8894dbd6739bd5a6e7: Status 404 returned error can't find the container with id 15afd821174604e88d650c6765f46aee0cc1e9cfb0b9fa8894dbd6739bd5a6e7
Jul 28 20:11:06.060891 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:06.060864    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-49sbr" event={"ID":"8131278e-1e59-4a56-83a2-ed9a0b882c98","Type":"ContainerStarted","Data":"15afd821174604e88d650c6765f46aee0cc1e9cfb0b9fa8894dbd6739bd5a6e7"}
Jul 28 20:11:07.067038 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:07.066998    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-49sbr" event={"ID":"8131278e-1e59-4a56-83a2-ed9a0b882c98","Type":"ContainerStarted","Data":"0b77a805b338eb4d7a4cd5615cf01169b77c9d50c603d9ecfb6a3748badcec7b"}
Jul 28 20:11:07.085003 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:07.084951    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-49sbr" podStartSLOduration=1.75830708 podStartE2EDuration="2.08493636s" totalImagesPullingTime="326.62928ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:11:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:11:07.083559469 +0000 UTC m=+729.866570911" watchObservedRunningTime="2026-07-28 20:11:07.08493636 +0000 UTC m=+729.867947847"
Jul 28 20:11:07.812642 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:07.812605    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zx8p7"
Jul 28 20:11:07.812820 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:07.812692    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zx8p7"
Jul 28 20:11:07.825289 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:07.825263    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zx8p7"
Jul 28 20:11:08.083151 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:08.083047    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zx8p7"
Jul 28 20:11:15.592176 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:15.592135    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-49sbr"
Jul 28 20:11:15.592635 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:15.592207    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-49sbr"
Jul 28 20:11:15.604676 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:15.604648    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-49sbr"
Jul 28 20:11:16.119410 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:16.119380    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-49sbr"
Jul 28 20:11:40.681213 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:40.681175    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-ktsv2"]
Jul 28 20:11:40.687374 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:40.687349    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ktsv2"
Jul 28 20:11:40.690036 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:40.690014    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 28 20:11:40.695038 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:40.695006    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-ktsv2"]
Jul 28 20:11:40.713525 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:40.713491    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c9d3df2d-be95-44cd-8d58-6f93fd9ed81b-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ktsv2\" (UID: \"c9d3df2d-be95-44cd-8d58-6f93fd9ed81b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ktsv2"
Jul 28 20:11:40.713668 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:40.713539    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9d3df2d-be95-44cd-8d58-6f93fd9ed81b-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ktsv2\" (UID: \"c9d3df2d-be95-44cd-8d58-6f93fd9ed81b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ktsv2"
Jul 28 20:11:40.713668 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:40.713561    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c9d3df2d-be95-44cd-8d58-6f93fd9ed81b-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ktsv2\" (UID: \"c9d3df2d-be95-44cd-8d58-6f93fd9ed81b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ktsv2"
Jul 28 20:11:40.713668 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:40.713630    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z52t6\" (UniqueName: \"kubernetes.io/projected/c9d3df2d-be95-44cd-8d58-6f93fd9ed81b-kube-api-access-z52t6\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ktsv2\" (UID: \"c9d3df2d-be95-44cd-8d58-6f93fd9ed81b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ktsv2"
Jul 28 20:11:40.713783 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:40.713685    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c9d3df2d-be95-44cd-8d58-6f93fd9ed81b-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ktsv2\" (UID: \"c9d3df2d-be95-44cd-8d58-6f93fd9ed81b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ktsv2"
Jul 28 20:11:40.713783 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:40.713746    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c9d3df2d-be95-44cd-8d58-6f93fd9ed81b-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ktsv2\" (UID: \"c9d3df2d-be95-44cd-8d58-6f93fd9ed81b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ktsv2"
Jul 28 20:11:40.814966 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:40.814929    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c9d3df2d-be95-44cd-8d58-6f93fd9ed81b-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ktsv2\" (UID: \"c9d3df2d-be95-44cd-8d58-6f93fd9ed81b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ktsv2"
Jul 28 20:11:40.814966 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:40.814975    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9d3df2d-be95-44cd-8d58-6f93fd9ed81b-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ktsv2\" (UID: \"c9d3df2d-be95-44cd-8d58-6f93fd9ed81b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ktsv2"
Jul 28 20:11:40.815219 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:40.814997    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c9d3df2d-be95-44cd-8d58-6f93fd9ed81b-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ktsv2\" (UID: \"c9d3df2d-be95-44cd-8d58-6f93fd9ed81b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ktsv2"
Jul 28 20:11:40.815219 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:40.815121    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z52t6\" (UniqueName: \"kubernetes.io/projected/c9d3df2d-be95-44cd-8d58-6f93fd9ed81b-kube-api-access-z52t6\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ktsv2\" (UID: \"c9d3df2d-be95-44cd-8d58-6f93fd9ed81b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ktsv2"
Jul 28 20:11:40.815299 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:40.815215    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c9d3df2d-be95-44cd-8d58-6f93fd9ed81b-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ktsv2\" (UID: \"c9d3df2d-be95-44cd-8d58-6f93fd9ed81b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ktsv2"
Jul 28 20:11:40.815342 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:40.815296    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c9d3df2d-be95-44cd-8d58-6f93fd9ed81b-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ktsv2\" (UID: \"c9d3df2d-be95-44cd-8d58-6f93fd9ed81b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ktsv2"
Jul 28 20:11:40.815423 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:40.815394    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c9d3df2d-be95-44cd-8d58-6f93fd9ed81b-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ktsv2\" (UID: \"c9d3df2d-be95-44cd-8d58-6f93fd9ed81b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ktsv2"
Jul 28 20:11:40.815538 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:40.815438    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9d3df2d-be95-44cd-8d58-6f93fd9ed81b-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ktsv2\" (UID: \"c9d3df2d-be95-44cd-8d58-6f93fd9ed81b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ktsv2"
Jul 28 20:11:40.815585 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:40.815542    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c9d3df2d-be95-44cd-8d58-6f93fd9ed81b-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ktsv2\" (UID: \"c9d3df2d-be95-44cd-8d58-6f93fd9ed81b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ktsv2"
Jul 28 20:11:40.817454 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:40.817434    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c9d3df2d-be95-44cd-8d58-6f93fd9ed81b-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ktsv2\" (UID: \"c9d3df2d-be95-44cd-8d58-6f93fd9ed81b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ktsv2"
Jul 28 20:11:40.817645 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:40.817626    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c9d3df2d-be95-44cd-8d58-6f93fd9ed81b-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ktsv2\" (UID: \"c9d3df2d-be95-44cd-8d58-6f93fd9ed81b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ktsv2"
Jul 28 20:11:40.823247 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:40.823229    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z52t6\" (UniqueName: \"kubernetes.io/projected/c9d3df2d-be95-44cd-8d58-6f93fd9ed81b-kube-api-access-z52t6\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ktsv2\" (UID: \"c9d3df2d-be95-44cd-8d58-6f93fd9ed81b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ktsv2"
Jul 28 20:11:40.999849 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:40.999760    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ktsv2"
Jul 28 20:11:41.135257 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:41.135230    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-ktsv2"]
Jul 28 20:11:41.136807 ip-10-0-139-54 kubenswrapper[2554]: W0728 20:11:41.136781    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9d3df2d_be95_44cd_8d58_6f93fd9ed81b.slice/crio-eff00320d6fb3645bb168befa05364d96a54dd4a81f9d9df3719764d616c4ce1 WatchSource:0}: Error finding container eff00320d6fb3645bb168befa05364d96a54dd4a81f9d9df3719764d616c4ce1: Status 404 returned error can't find the container with id eff00320d6fb3645bb168befa05364d96a54dd4a81f9d9df3719764d616c4ce1
Jul 28 20:11:41.222415 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:41.222379    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ktsv2" event={"ID":"c9d3df2d-be95-44cd-8d58-6f93fd9ed81b","Type":"ContainerStarted","Data":"eff00320d6fb3645bb168befa05364d96a54dd4a81f9d9df3719764d616c4ce1"}
Jul 28 20:11:42.228576 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:42.228536    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ktsv2" event={"ID":"c9d3df2d-be95-44cd-8d58-6f93fd9ed81b","Type":"ContainerStarted","Data":"1403d8d9c800a3c0e51f40b49ae8541cda9f21d8a9aa3be533b69d1a8a618958"}
Jul 28 20:11:42.245307 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:42.245254    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ktsv2" podStartSLOduration=2.054404554 podStartE2EDuration="2.245237747s" totalImagesPullingTime="190.833193ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:11:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:11:42.2448939 +0000 UTC m=+765.027905342" watchObservedRunningTime="2026-07-28 20:11:42.245237747 +0000 UTC m=+765.028249189"
Jul 28 20:11:51.000141 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:51.000059    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ktsv2"
Jul 28 20:11:51.000611 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:51.000192    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ktsv2"
Jul 28 20:11:51.012861 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:51.012838    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ktsv2"
Jul 28 20:11:51.280478 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:51.280399    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ktsv2"
Jul 28 20:11:54.184338 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:54.184303    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-tvqhc"]
Jul 28 20:11:54.187573 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:54.187555    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tvqhc"
Jul 28 20:11:54.190254 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:54.190225    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 28 20:11:54.196793 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:54.196774    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-tvqhc"]
Jul 28 20:11:54.239000 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:54.238965    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e29b582c-3dc7-4e0e-af68-83f39cbbfc0d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tvqhc\" (UID: \"e29b582c-3dc7-4e0e-af68-83f39cbbfc0d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tvqhc"
Jul 28 20:11:54.239000 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:54.239001    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e29b582c-3dc7-4e0e-af68-83f39cbbfc0d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tvqhc\" (UID: \"e29b582c-3dc7-4e0e-af68-83f39cbbfc0d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tvqhc"
Jul 28 20:11:54.239290 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:54.239027    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e29b582c-3dc7-4e0e-af68-83f39cbbfc0d-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tvqhc\" (UID: \"e29b582c-3dc7-4e0e-af68-83f39cbbfc0d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tvqhc"
Jul 28 20:11:54.239290 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:54.239101    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e29b582c-3dc7-4e0e-af68-83f39cbbfc0d-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tvqhc\" (UID: \"e29b582c-3dc7-4e0e-af68-83f39cbbfc0d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tvqhc"
Jul 28 20:11:54.239290 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:54.239183    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95z47\" (UniqueName: \"kubernetes.io/projected/e29b582c-3dc7-4e0e-af68-83f39cbbfc0d-kube-api-access-95z47\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tvqhc\" (UID: \"e29b582c-3dc7-4e0e-af68-83f39cbbfc0d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tvqhc"
Jul 28 20:11:54.239290 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:54.239284    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e29b582c-3dc7-4e0e-af68-83f39cbbfc0d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tvqhc\" (UID: \"e29b582c-3dc7-4e0e-af68-83f39cbbfc0d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tvqhc"
Jul 28 20:11:54.339895 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:54.339837    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95z47\" (UniqueName: \"kubernetes.io/projected/e29b582c-3dc7-4e0e-af68-83f39cbbfc0d-kube-api-access-95z47\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tvqhc\" (UID: \"e29b582c-3dc7-4e0e-af68-83f39cbbfc0d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tvqhc"
Jul 28 20:11:54.340127 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:54.339931    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e29b582c-3dc7-4e0e-af68-83f39cbbfc0d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tvqhc\" (UID: \"e29b582c-3dc7-4e0e-af68-83f39cbbfc0d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tvqhc"
Jul 28 20:11:54.340127 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:54.339958    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e29b582c-3dc7-4e0e-af68-83f39cbbfc0d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tvqhc\" (UID: \"e29b582c-3dc7-4e0e-af68-83f39cbbfc0d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tvqhc"
Jul 28 20:11:54.340127 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:54.339977    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e29b582c-3dc7-4e0e-af68-83f39cbbfc0d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tvqhc\" (UID: \"e29b582c-3dc7-4e0e-af68-83f39cbbfc0d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tvqhc"
Jul 28 20:11:54.340127 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:54.340002    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e29b582c-3dc7-4e0e-af68-83f39cbbfc0d-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tvqhc\" (UID: \"e29b582c-3dc7-4e0e-af68-83f39cbbfc0d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tvqhc"
Jul 28 20:11:54.340127 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:54.340025    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e29b582c-3dc7-4e0e-af68-83f39cbbfc0d-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tvqhc\" (UID: \"e29b582c-3dc7-4e0e-af68-83f39cbbfc0d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tvqhc"
Jul 28 20:11:54.340473 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:54.340444    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e29b582c-3dc7-4e0e-af68-83f39cbbfc0d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tvqhc\" (UID: \"e29b582c-3dc7-4e0e-af68-83f39cbbfc0d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tvqhc"
Jul 28 20:11:54.340530 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:54.340452    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e29b582c-3dc7-4e0e-af68-83f39cbbfc0d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tvqhc\" (UID: \"e29b582c-3dc7-4e0e-af68-83f39cbbfc0d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tvqhc"
Jul 28 20:11:54.340530 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:54.340494    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e29b582c-3dc7-4e0e-af68-83f39cbbfc0d-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tvqhc\" (UID: \"e29b582c-3dc7-4e0e-af68-83f39cbbfc0d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tvqhc"
Jul 28 20:11:54.342284 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:54.342266    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e29b582c-3dc7-4e0e-af68-83f39cbbfc0d-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tvqhc\" (UID: \"e29b582c-3dc7-4e0e-af68-83f39cbbfc0d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tvqhc"
Jul 28 20:11:54.342554 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:54.342538    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e29b582c-3dc7-4e0e-af68-83f39cbbfc0d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tvqhc\" (UID: \"e29b582c-3dc7-4e0e-af68-83f39cbbfc0d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tvqhc"
Jul 28 20:11:54.347596 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:54.347574    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95z47\" (UniqueName: \"kubernetes.io/projected/e29b582c-3dc7-4e0e-af68-83f39cbbfc0d-kube-api-access-95z47\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tvqhc\" (UID: \"e29b582c-3dc7-4e0e-af68-83f39cbbfc0d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tvqhc"
Jul 28 20:11:54.498538 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:54.498446    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tvqhc"
Jul 28 20:11:54.629032 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:54.629009    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-tvqhc"]
Jul 28 20:11:54.631106 ip-10-0-139-54 kubenswrapper[2554]: W0728 20:11:54.631054    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode29b582c_3dc7_4e0e_af68_83f39cbbfc0d.slice/crio-68e523ebfaf41c795145bbe6d4db26b6230f5229ea34acc075c867c79a467e44 WatchSource:0}: Error finding container 68e523ebfaf41c795145bbe6d4db26b6230f5229ea34acc075c867c79a467e44: Status 404 returned error can't find the container with id 68e523ebfaf41c795145bbe6d4db26b6230f5229ea34acc075c867c79a467e44
Jul 28 20:11:55.287604 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:55.287570    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tvqhc" event={"ID":"e29b582c-3dc7-4e0e-af68-83f39cbbfc0d","Type":"ContainerStarted","Data":"a5318452eb7cf97648fa6db81c3b1d84e1ce7109e81e7d15b53dedce25f509d8"}
Jul 28 20:11:55.287944 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:55.287612    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tvqhc" event={"ID":"e29b582c-3dc7-4e0e-af68-83f39cbbfc0d","Type":"ContainerStarted","Data":"68e523ebfaf41c795145bbe6d4db26b6230f5229ea34acc075c867c79a467e44"}
Jul 28 20:11:55.306341 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:11:55.306281    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tvqhc" podStartSLOduration=1.104916744 podStartE2EDuration="1.306265108s" totalImagesPullingTime="201.348364ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:11:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:11:55.304606767 +0000 UTC m=+778.087618209" watchObservedRunningTime="2026-07-28 20:11:55.306265108 +0000 UTC m=+778.089276586"
Jul 28 20:12:04.499493 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:12:04.499446    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tvqhc"
Jul 28 20:12:04.499875 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:12:04.499511    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tvqhc"
Jul 28 20:12:04.511823 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:12:04.511797    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tvqhc"
Jul 28 20:12:05.343181 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:12:05.343153    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tvqhc"
Jul 28 20:13:57.737217 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:13:57.737143    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-7vcr8_dde4c1a8-8229-4419-87b6-d8e39a0827c4/console-operator/1.log"
Jul 28 20:13:57.738595 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:13:57.738570    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-7vcr8_dde4c1a8-8229-4419-87b6-d8e39a0827c4/console-operator/1.log"
Jul 28 20:18:57.766140 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:18:57.766111    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-7vcr8_dde4c1a8-8229-4419-87b6-d8e39a0827c4/console-operator/1.log"
Jul 28 20:18:57.766772 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:18:57.766671    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-7vcr8_dde4c1a8-8229-4419-87b6-d8e39a0827c4/console-operator/1.log"
Jul 28 20:23:57.794835 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:23:57.794807    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-7vcr8_dde4c1a8-8229-4419-87b6-d8e39a0827c4/console-operator/1.log"
Jul 28 20:23:57.797863 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:23:57.797834    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-7vcr8_dde4c1a8-8229-4419-87b6-d8e39a0827c4/console-operator/1.log"
Jul 28 20:28:57.822126 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:28:57.822099    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-7vcr8_dde4c1a8-8229-4419-87b6-d8e39a0827c4/console-operator/1.log"
Jul 28 20:28:57.825892 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:28:57.825867    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-7vcr8_dde4c1a8-8229-4419-87b6-d8e39a0827c4/console-operator/1.log"
Jul 28 20:33:57.849597 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:33:57.849566    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-7vcr8_dde4c1a8-8229-4419-87b6-d8e39a0827c4/console-operator/1.log"
Jul 28 20:33:57.852783 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:33:57.852763    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-7vcr8_dde4c1a8-8229-4419-87b6-d8e39a0827c4/console-operator/1.log"
Jul 28 20:38:57.879107 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:38:57.877717    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-7vcr8_dde4c1a8-8229-4419-87b6-d8e39a0827c4/console-operator/1.log"
Jul 28 20:38:57.882886 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:38:57.882859    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-7vcr8_dde4c1a8-8229-4419-87b6-d8e39a0827c4/console-operator/1.log"
Jul 28 20:39:48.089027 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:39:48.088949    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-d38c0f/test-model-d38c0f-kserve-746dc64877-jfsb6"]
Jul 28 20:39:48.092464 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:39:48.092441    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-d38c0f/test-model-d38c0f-kserve-746dc64877-jfsb6"
Jul 28 20:39:48.097093 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:39:48.097047    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-d38c0f\"/\"kube-root-ca.crt\""
Jul 28 20:39:48.098418 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:39:48.098391    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-d38c0f\"/\"test-model-d38c0f-kserve-self-signed-certs\""
Jul 28 20:39:48.098548 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:39:48.098535    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-d38c0f\"/\"openshift-service-ca.crt\""
Jul 28 20:39:48.098612 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:39:48.098576    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-d38c0f\"/\"default-dockercfg-mfnw8\""
Jul 28 20:39:48.111246 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:39:48.111220    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-d38c0f/test-model-d38c0f-kserve-746dc64877-jfsb6"]
Jul 28 20:39:48.190391 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:39:48.190346    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f58af6e-ddef-40f4-9487-6dac56c9a224-model-cache\") pod \"test-model-d38c0f-kserve-746dc64877-jfsb6\" (UID: \"1f58af6e-ddef-40f4-9487-6dac56c9a224\") " pod="ai-tenant-e2e-inf-b-d38c0f/test-model-d38c0f-kserve-746dc64877-jfsb6"
Jul 28 20:39:48.190391 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:39:48.190386    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f58af6e-ddef-40f4-9487-6dac56c9a224-tls-certs\") pod \"test-model-d38c0f-kserve-746dc64877-jfsb6\" (UID: \"1f58af6e-ddef-40f4-9487-6dac56c9a224\") " pod="ai-tenant-e2e-inf-b-d38c0f/test-model-d38c0f-kserve-746dc64877-jfsb6"
Jul 28 20:39:48.190586 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:39:48.190443    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjjbw\" (UniqueName: \"kubernetes.io/projected/1f58af6e-ddef-40f4-9487-6dac56c9a224-kube-api-access-kjjbw\") pod \"test-model-d38c0f-kserve-746dc64877-jfsb6\" (UID: \"1f58af6e-ddef-40f4-9487-6dac56c9a224\") " pod="ai-tenant-e2e-inf-b-d38c0f/test-model-d38c0f-kserve-746dc64877-jfsb6"
Jul 28 20:39:48.190586 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:39:48.190478    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f58af6e-ddef-40f4-9487-6dac56c9a224-home\") pod \"test-model-d38c0f-kserve-746dc64877-jfsb6\" (UID: \"1f58af6e-ddef-40f4-9487-6dac56c9a224\") " pod="ai-tenant-e2e-inf-b-d38c0f/test-model-d38c0f-kserve-746dc64877-jfsb6"
Jul 28 20:39:48.190656 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:39:48.190579    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f58af6e-ddef-40f4-9487-6dac56c9a224-tmp-dir\") pod \"test-model-d38c0f-kserve-746dc64877-jfsb6\" (UID: \"1f58af6e-ddef-40f4-9487-6dac56c9a224\") " pod="ai-tenant-e2e-inf-b-d38c0f/test-model-d38c0f-kserve-746dc64877-jfsb6"
Jul 28 20:39:48.190656 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:39:48.190637    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f58af6e-ddef-40f4-9487-6dac56c9a224-dshm\") pod \"test-model-d38c0f-kserve-746dc64877-jfsb6\" (UID: \"1f58af6e-ddef-40f4-9487-6dac56c9a224\") " pod="ai-tenant-e2e-inf-b-d38c0f/test-model-d38c0f-kserve-746dc64877-jfsb6"
Jul 28 20:39:48.291526 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:39:48.291487    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f58af6e-ddef-40f4-9487-6dac56c9a224-dshm\") pod \"test-model-d38c0f-kserve-746dc64877-jfsb6\" (UID: \"1f58af6e-ddef-40f4-9487-6dac56c9a224\") " pod="ai-tenant-e2e-inf-b-d38c0f/test-model-d38c0f-kserve-746dc64877-jfsb6"
Jul 28 20:39:48.291526 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:39:48.291527    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f58af6e-ddef-40f4-9487-6dac56c9a224-model-cache\") pod \"test-model-d38c0f-kserve-746dc64877-jfsb6\" (UID: \"1f58af6e-ddef-40f4-9487-6dac56c9a224\") " pod="ai-tenant-e2e-inf-b-d38c0f/test-model-d38c0f-kserve-746dc64877-jfsb6"
Jul 28 20:39:48.291774 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:39:48.291544    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f58af6e-ddef-40f4-9487-6dac56c9a224-tls-certs\") pod \"test-model-d38c0f-kserve-746dc64877-jfsb6\" (UID: \"1f58af6e-ddef-40f4-9487-6dac56c9a224\") " pod="ai-tenant-e2e-inf-b-d38c0f/test-model-d38c0f-kserve-746dc64877-jfsb6"
Jul 28 20:39:48.291774 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:39:48.291575    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjjbw\" (UniqueName: \"kubernetes.io/projected/1f58af6e-ddef-40f4-9487-6dac56c9a224-kube-api-access-kjjbw\") pod \"test-model-d38c0f-kserve-746dc64877-jfsb6\" (UID: \"1f58af6e-ddef-40f4-9487-6dac56c9a224\") " pod="ai-tenant-e2e-inf-b-d38c0f/test-model-d38c0f-kserve-746dc64877-jfsb6"
Jul 28 20:39:48.291774 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:39:48.291614    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f58af6e-ddef-40f4-9487-6dac56c9a224-home\") pod \"test-model-d38c0f-kserve-746dc64877-jfsb6\" (UID: \"1f58af6e-ddef-40f4-9487-6dac56c9a224\") " pod="ai-tenant-e2e-inf-b-d38c0f/test-model-d38c0f-kserve-746dc64877-jfsb6"
Jul 28 20:39:48.291774 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:39:48.291667    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f58af6e-ddef-40f4-9487-6dac56c9a224-tmp-dir\") pod \"test-model-d38c0f-kserve-746dc64877-jfsb6\" (UID: \"1f58af6e-ddef-40f4-9487-6dac56c9a224\") " pod="ai-tenant-e2e-inf-b-d38c0f/test-model-d38c0f-kserve-746dc64877-jfsb6"
Jul 28 20:39:48.291982 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:39:48.291944    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f58af6e-ddef-40f4-9487-6dac56c9a224-model-cache\") pod \"test-model-d38c0f-kserve-746dc64877-jfsb6\" (UID: \"1f58af6e-ddef-40f4-9487-6dac56c9a224\") " pod="ai-tenant-e2e-inf-b-d38c0f/test-model-d38c0f-kserve-746dc64877-jfsb6"
Jul 28 20:39:48.292042 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:39:48.292006    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f58af6e-ddef-40f4-9487-6dac56c9a224-home\") pod \"test-model-d38c0f-kserve-746dc64877-jfsb6\" (UID: \"1f58af6e-ddef-40f4-9487-6dac56c9a224\") " pod="ai-tenant-e2e-inf-b-d38c0f/test-model-d38c0f-kserve-746dc64877-jfsb6"
Jul 28 20:39:48.292129 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:39:48.292051    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f58af6e-ddef-40f4-9487-6dac56c9a224-tmp-dir\") pod \"test-model-d38c0f-kserve-746dc64877-jfsb6\" (UID: \"1f58af6e-ddef-40f4-9487-6dac56c9a224\") " pod="ai-tenant-e2e-inf-b-d38c0f/test-model-d38c0f-kserve-746dc64877-jfsb6"
Jul 28 20:39:48.293796 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:39:48.293777    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f58af6e-ddef-40f4-9487-6dac56c9a224-dshm\") pod \"test-model-d38c0f-kserve-746dc64877-jfsb6\" (UID: \"1f58af6e-ddef-40f4-9487-6dac56c9a224\") " pod="ai-tenant-e2e-inf-b-d38c0f/test-model-d38c0f-kserve-746dc64877-jfsb6"
Jul 28 20:39:48.293974 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:39:48.293957    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f58af6e-ddef-40f4-9487-6dac56c9a224-tls-certs\") pod \"test-model-d38c0f-kserve-746dc64877-jfsb6\" (UID: \"1f58af6e-ddef-40f4-9487-6dac56c9a224\") " pod="ai-tenant-e2e-inf-b-d38c0f/test-model-d38c0f-kserve-746dc64877-jfsb6"
Jul 28 20:39:48.303771 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:39:48.303745    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjjbw\" (UniqueName: \"kubernetes.io/projected/1f58af6e-ddef-40f4-9487-6dac56c9a224-kube-api-access-kjjbw\") pod \"test-model-d38c0f-kserve-746dc64877-jfsb6\" (UID: \"1f58af6e-ddef-40f4-9487-6dac56c9a224\") " pod="ai-tenant-e2e-inf-b-d38c0f/test-model-d38c0f-kserve-746dc64877-jfsb6"
Jul 28 20:39:48.402000 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:39:48.401909    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-d38c0f/test-model-d38c0f-kserve-746dc64877-jfsb6"
Jul 28 20:39:48.540064 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:39:48.540036    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-d38c0f/test-model-d38c0f-kserve-746dc64877-jfsb6"]
Jul 28 20:39:48.542098 ip-10-0-139-54 kubenswrapper[2554]: W0728 20:39:48.542052    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1f58af6e_ddef_40f4_9487_6dac56c9a224.slice/crio-2846430e77f0937e8e786d7eb6c60797b1ce6c08de75d2a88c2656f3d9747ca4 WatchSource:0}: Error finding container 2846430e77f0937e8e786d7eb6c60797b1ce6c08de75d2a88c2656f3d9747ca4: Status 404 returned error can't find the container with id 2846430e77f0937e8e786d7eb6c60797b1ce6c08de75d2a88c2656f3d9747ca4
Jul 28 20:39:48.835065 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:39:48.835025    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-d38c0f/test-model-d38c0f-kserve-746dc64877-jfsb6" event={"ID":"1f58af6e-ddef-40f4-9487-6dac56c9a224","Type":"ContainerStarted","Data":"76883944d838857dcf5bbd87e1ee8a662e34c2ca75082d65b0a66ca5511ebc53"}
Jul 28 20:39:48.835254 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:39:48.835101    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-d38c0f/test-model-d38c0f-kserve-746dc64877-jfsb6" event={"ID":"1f58af6e-ddef-40f4-9487-6dac56c9a224","Type":"ContainerStarted","Data":"2846430e77f0937e8e786d7eb6c60797b1ce6c08de75d2a88c2656f3d9747ca4"}
Jul 28 20:39:48.856112 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:39:48.856040    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-d38c0f/test-model-d38c0f-kserve-746dc64877-jfsb6" podStartSLOduration=0.856024422 podStartE2EDuration="856.024422ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:39:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:39:48.854948727 +0000 UTC m=+2451.637960170" watchObservedRunningTime="2026-07-28 20:39:48.856024422 +0000 UTC m=+2451.639035865"
Jul 28 20:39:58.402978 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:39:58.402916    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-d38c0f/test-model-d38c0f-kserve-746dc64877-jfsb6"
Jul 28 20:39:58.402978 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:39:58.402994    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-d38c0f/test-model-d38c0f-kserve-746dc64877-jfsb6"
Jul 28 20:39:58.415356 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:39:58.415323    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-d38c0f/test-model-d38c0f-kserve-746dc64877-jfsb6"
Jul 28 20:39:58.888789 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:39:58.888760    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-d38c0f/test-model-d38c0f-kserve-746dc64877-jfsb6"
Jul 28 20:40:41.456179 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:40:41.456144    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-d38c0f/test-model-d38c0f-kserve-746dc64877-jfsb6"]
Jul 28 20:40:56.475187 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:40:56.475067    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-d38c0f/test-model-d38c0f-kserve-746dc64877-jfsb6" podUID="1f58af6e-ddef-40f4-9487-6dac56c9a224" containerName="main" containerID="cri-o://76883944d838857dcf5bbd87e1ee8a662e34c2ca75082d65b0a66ca5511ebc53" gracePeriod=45
Jul 28 20:40:56.725820 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:40:56.725761    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-d38c0f/test-model-d38c0f-kserve-746dc64877-jfsb6"
Jul 28 20:40:56.817158 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:40:56.817115    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kjjbw\" (UniqueName: \"kubernetes.io/projected/1f58af6e-ddef-40f4-9487-6dac56c9a224-kube-api-access-kjjbw\") pod \"1f58af6e-ddef-40f4-9487-6dac56c9a224\" (UID: \"1f58af6e-ddef-40f4-9487-6dac56c9a224\") "
Jul 28 20:40:56.817158 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:40:56.817157    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f58af6e-ddef-40f4-9487-6dac56c9a224-tmp-dir\") pod \"1f58af6e-ddef-40f4-9487-6dac56c9a224\" (UID: \"1f58af6e-ddef-40f4-9487-6dac56c9a224\") "
Jul 28 20:40:56.817393 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:40:56.817195    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f58af6e-ddef-40f4-9487-6dac56c9a224-home\") pod \"1f58af6e-ddef-40f4-9487-6dac56c9a224\" (UID: \"1f58af6e-ddef-40f4-9487-6dac56c9a224\") "
Jul 28 20:40:56.817393 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:40:56.817227    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f58af6e-ddef-40f4-9487-6dac56c9a224-tls-certs\") pod \"1f58af6e-ddef-40f4-9487-6dac56c9a224\" (UID: \"1f58af6e-ddef-40f4-9487-6dac56c9a224\") "
Jul 28 20:40:56.817393 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:40:56.817287    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f58af6e-ddef-40f4-9487-6dac56c9a224-model-cache\") pod \"1f58af6e-ddef-40f4-9487-6dac56c9a224\" (UID: \"1f58af6e-ddef-40f4-9487-6dac56c9a224\") "
Jul 28 20:40:56.817393 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:40:56.817335    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f58af6e-ddef-40f4-9487-6dac56c9a224-dshm\") pod \"1f58af6e-ddef-40f4-9487-6dac56c9a224\" (UID: \"1f58af6e-ddef-40f4-9487-6dac56c9a224\") "
Jul 28 20:40:56.817598 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:40:56.817452    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f58af6e-ddef-40f4-9487-6dac56c9a224-home" (OuterVolumeSpecName: "home") pod "1f58af6e-ddef-40f4-9487-6dac56c9a224" (UID: "1f58af6e-ddef-40f4-9487-6dac56c9a224"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:40:56.817598 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:40:56.817464    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f58af6e-ddef-40f4-9487-6dac56c9a224-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1f58af6e-ddef-40f4-9487-6dac56c9a224" (UID: "1f58af6e-ddef-40f4-9487-6dac56c9a224"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:40:56.817598 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:40:56.817554    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f58af6e-ddef-40f4-9487-6dac56c9a224-model-cache" (OuterVolumeSpecName: "model-cache") pod "1f58af6e-ddef-40f4-9487-6dac56c9a224" (UID: "1f58af6e-ddef-40f4-9487-6dac56c9a224"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:40:56.817728 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:40:56.817706    2554 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f58af6e-ddef-40f4-9487-6dac56c9a224-tmp-dir\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:40:56.817728 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:40:56.817724    2554 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f58af6e-ddef-40f4-9487-6dac56c9a224-home\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:40:56.817791 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:40:56.817736    2554 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f58af6e-ddef-40f4-9487-6dac56c9a224-model-cache\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:40:56.819472 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:40:56.819441    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f58af6e-ddef-40f4-9487-6dac56c9a224-kube-api-access-kjjbw" (OuterVolumeSpecName: "kube-api-access-kjjbw") pod "1f58af6e-ddef-40f4-9487-6dac56c9a224" (UID: "1f58af6e-ddef-40f4-9487-6dac56c9a224"). InnerVolumeSpecName "kube-api-access-kjjbw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:40:56.819472 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:40:56.819428    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f58af6e-ddef-40f4-9487-6dac56c9a224-dshm" (OuterVolumeSpecName: "dshm") pod "1f58af6e-ddef-40f4-9487-6dac56c9a224" (UID: "1f58af6e-ddef-40f4-9487-6dac56c9a224"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:40:56.819613 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:40:56.819519    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f58af6e-ddef-40f4-9487-6dac56c9a224-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1f58af6e-ddef-40f4-9487-6dac56c9a224" (UID: "1f58af6e-ddef-40f4-9487-6dac56c9a224"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:40:56.918990 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:40:56.918947    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kjjbw\" (UniqueName: \"kubernetes.io/projected/1f58af6e-ddef-40f4-9487-6dac56c9a224-kube-api-access-kjjbw\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:40:56.918990 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:40:56.918978    2554 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f58af6e-ddef-40f4-9487-6dac56c9a224-tls-certs\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:40:56.918990 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:40:56.918996    2554 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f58af6e-ddef-40f4-9487-6dac56c9a224-dshm\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:40:57.142968 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:40:57.142928    2554 generic.go:356] "Generic (PLEG): container finished" podID="1f58af6e-ddef-40f4-9487-6dac56c9a224" containerID="76883944d838857dcf5bbd87e1ee8a662e34c2ca75082d65b0a66ca5511ebc53" exitCode=0
Jul 28 20:40:57.142968 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:40:57.142973    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-d38c0f/test-model-d38c0f-kserve-746dc64877-jfsb6"
Jul 28 20:40:57.143251 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:40:57.143013    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-d38c0f/test-model-d38c0f-kserve-746dc64877-jfsb6" event={"ID":"1f58af6e-ddef-40f4-9487-6dac56c9a224","Type":"ContainerDied","Data":"76883944d838857dcf5bbd87e1ee8a662e34c2ca75082d65b0a66ca5511ebc53"}
Jul 28 20:40:57.143251 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:40:57.143063    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-d38c0f/test-model-d38c0f-kserve-746dc64877-jfsb6" event={"ID":"1f58af6e-ddef-40f4-9487-6dac56c9a224","Type":"ContainerDied","Data":"2846430e77f0937e8e786d7eb6c60797b1ce6c08de75d2a88c2656f3d9747ca4"}
Jul 28 20:40:57.143251 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:40:57.143098    2554 scope.go:117] "RemoveContainer" containerID="76883944d838857dcf5bbd87e1ee8a662e34c2ca75082d65b0a66ca5511ebc53"
Jul 28 20:40:57.152111 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:40:57.152091    2554 scope.go:117] "RemoveContainer" containerID="76883944d838857dcf5bbd87e1ee8a662e34c2ca75082d65b0a66ca5511ebc53"
Jul 28 20:40:57.152405 ip-10-0-139-54 kubenswrapper[2554]: E0728 20:40:57.152383    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76883944d838857dcf5bbd87e1ee8a662e34c2ca75082d65b0a66ca5511ebc53\": container with ID starting with 76883944d838857dcf5bbd87e1ee8a662e34c2ca75082d65b0a66ca5511ebc53 not found: ID does not exist" containerID="76883944d838857dcf5bbd87e1ee8a662e34c2ca75082d65b0a66ca5511ebc53"
Jul 28 20:40:57.152441 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:40:57.152416    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76883944d838857dcf5bbd87e1ee8a662e34c2ca75082d65b0a66ca5511ebc53"} err="failed to get container status \"76883944d838857dcf5bbd87e1ee8a662e34c2ca75082d65b0a66ca5511ebc53\": rpc error: code = NotFound desc = could not find container \"76883944d838857dcf5bbd87e1ee8a662e34c2ca75082d65b0a66ca5511ebc53\": container with ID starting with 76883944d838857dcf5bbd87e1ee8a662e34c2ca75082d65b0a66ca5511ebc53 not found: ID does not exist"
Jul 28 20:40:57.165857 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:40:57.165826    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-d38c0f/test-model-d38c0f-kserve-746dc64877-jfsb6"]
Jul 28 20:40:57.169437 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:40:57.169413    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-d38c0f/test-model-d38c0f-kserve-746dc64877-jfsb6"]
Jul 28 20:40:57.777682 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:40:57.777648    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f58af6e-ddef-40f4-9487-6dac56c9a224" path="/var/lib/kubelet/pods/1f58af6e-ddef-40f4-9487-6dac56c9a224/volumes"
Jul 28 20:43:57.912197 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:43:57.912112    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-7vcr8_dde4c1a8-8229-4419-87b6-d8e39a0827c4/console-operator/1.log"
Jul 28 20:43:57.918040 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:43:57.918014    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-7vcr8_dde4c1a8-8229-4419-87b6-d8e39a0827c4/console-operator/1.log"
Jul 28 20:44:49.610919 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:44:49.610881    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-c0e826/auth-test-model-c0e826-kserve-58445f9db7-vtnvd"]
Jul 28 20:44:49.611529 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:44:49.611243    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f58af6e-ddef-40f4-9487-6dac56c9a224" containerName="main"
Jul 28 20:44:49.611529 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:44:49.611255    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f58af6e-ddef-40f4-9487-6dac56c9a224" containerName="main"
Jul 28 20:44:49.611529 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:44:49.611316    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f58af6e-ddef-40f4-9487-6dac56c9a224" containerName="main"
Jul 28 20:44:49.614173 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:44:49.614153    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c0e826/auth-test-model-c0e826-kserve-58445f9db7-vtnvd"
Jul 28 20:44:49.618538 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:44:49.618510    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c0e826\"/\"kube-root-ca.crt\""
Jul 28 20:44:49.618631 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:44:49.618550    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd-tmp-dir\") pod \"auth-test-model-c0e826-kserve-58445f9db7-vtnvd\" (UID: \"aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd\") " pod="ai-tenant-e2e-shared-b-c0e826/auth-test-model-c0e826-kserve-58445f9db7-vtnvd"
Jul 28 20:44:49.618631 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:44:49.618593    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd-tls-certs\") pod \"auth-test-model-c0e826-kserve-58445f9db7-vtnvd\" (UID: \"aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd\") " pod="ai-tenant-e2e-shared-b-c0e826/auth-test-model-c0e826-kserve-58445f9db7-vtnvd"
Jul 28 20:44:49.618631 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:44:49.618620    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6n4f7\" (UniqueName: \"kubernetes.io/projected/aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd-kube-api-access-6n4f7\") pod \"auth-test-model-c0e826-kserve-58445f9db7-vtnvd\" (UID: \"aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd\") " pod="ai-tenant-e2e-shared-b-c0e826/auth-test-model-c0e826-kserve-58445f9db7-vtnvd"
Jul 28 20:44:49.618773 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:44:49.618673    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd-home\") pod \"auth-test-model-c0e826-kserve-58445f9db7-vtnvd\" (UID: \"aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd\") " pod="ai-tenant-e2e-shared-b-c0e826/auth-test-model-c0e826-kserve-58445f9db7-vtnvd"
Jul 28 20:44:49.618773 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:44:49.618740    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd-dshm\") pod \"auth-test-model-c0e826-kserve-58445f9db7-vtnvd\" (UID: \"aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd\") " pod="ai-tenant-e2e-shared-b-c0e826/auth-test-model-c0e826-kserve-58445f9db7-vtnvd"
Jul 28 20:44:49.618877 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:44:49.618834    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd-model-cache\") pod \"auth-test-model-c0e826-kserve-58445f9db7-vtnvd\" (UID: \"aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd\") " pod="ai-tenant-e2e-shared-b-c0e826/auth-test-model-c0e826-kserve-58445f9db7-vtnvd"
Jul 28 20:44:49.619892 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:44:49.619833    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c0e826\"/\"openshift-service-ca.crt\""
Jul 28 20:44:49.619988 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:44:49.619901    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c0e826\"/\"auth-test-model-c0e826-kserve-self-signed-certs\""
Jul 28 20:44:49.619988 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:44:49.619925    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c0e826\"/\"default-dockercfg-w78mt\""
Jul 28 20:44:49.629238 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:44:49.629215    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c0e826/auth-test-model-c0e826-kserve-58445f9db7-vtnvd"]
Jul 28 20:44:49.719412 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:44:49.719372    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd-tmp-dir\") pod \"auth-test-model-c0e826-kserve-58445f9db7-vtnvd\" (UID: \"aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd\") " pod="ai-tenant-e2e-shared-b-c0e826/auth-test-model-c0e826-kserve-58445f9db7-vtnvd"
Jul 28 20:44:49.719599 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:44:49.719420    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd-tls-certs\") pod \"auth-test-model-c0e826-kserve-58445f9db7-vtnvd\" (UID: \"aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd\") " pod="ai-tenant-e2e-shared-b-c0e826/auth-test-model-c0e826-kserve-58445f9db7-vtnvd"
Jul 28 20:44:49.719599 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:44:49.719446    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6n4f7\" (UniqueName: \"kubernetes.io/projected/aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd-kube-api-access-6n4f7\") pod \"auth-test-model-c0e826-kserve-58445f9db7-vtnvd\" (UID: \"aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd\") " pod="ai-tenant-e2e-shared-b-c0e826/auth-test-model-c0e826-kserve-58445f9db7-vtnvd"
Jul 28 20:44:49.719599 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:44:49.719480    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd-home\") pod \"auth-test-model-c0e826-kserve-58445f9db7-vtnvd\" (UID: \"aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd\") " pod="ai-tenant-e2e-shared-b-c0e826/auth-test-model-c0e826-kserve-58445f9db7-vtnvd"
Jul 28 20:44:49.719599 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:44:49.719502    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd-dshm\") pod \"auth-test-model-c0e826-kserve-58445f9db7-vtnvd\" (UID: \"aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd\") " pod="ai-tenant-e2e-shared-b-c0e826/auth-test-model-c0e826-kserve-58445f9db7-vtnvd"
Jul 28 20:44:49.719599 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:44:49.719559    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd-model-cache\") pod \"auth-test-model-c0e826-kserve-58445f9db7-vtnvd\" (UID: \"aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd\") " pod="ai-tenant-e2e-shared-b-c0e826/auth-test-model-c0e826-kserve-58445f9db7-vtnvd"
Jul 28 20:44:49.719850 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:44:49.719811    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd-tmp-dir\") pod \"auth-test-model-c0e826-kserve-58445f9db7-vtnvd\" (UID: \"aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd\") " pod="ai-tenant-e2e-shared-b-c0e826/auth-test-model-c0e826-kserve-58445f9db7-vtnvd"
Jul 28 20:44:49.719850 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:44:49.719825    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd-home\") pod \"auth-test-model-c0e826-kserve-58445f9db7-vtnvd\" (UID: \"aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd\") " pod="ai-tenant-e2e-shared-b-c0e826/auth-test-model-c0e826-kserve-58445f9db7-vtnvd"
Jul 28 20:44:49.719929 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:44:49.719884    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd-model-cache\") pod \"auth-test-model-c0e826-kserve-58445f9db7-vtnvd\" (UID: \"aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd\") " pod="ai-tenant-e2e-shared-b-c0e826/auth-test-model-c0e826-kserve-58445f9db7-vtnvd"
Jul 28 20:44:49.721857 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:44:49.721841    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd-dshm\") pod \"auth-test-model-c0e826-kserve-58445f9db7-vtnvd\" (UID: \"aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd\") " pod="ai-tenant-e2e-shared-b-c0e826/auth-test-model-c0e826-kserve-58445f9db7-vtnvd"
Jul 28 20:44:49.722002 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:44:49.721986    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd-tls-certs\") pod \"auth-test-model-c0e826-kserve-58445f9db7-vtnvd\" (UID: \"aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd\") " pod="ai-tenant-e2e-shared-b-c0e826/auth-test-model-c0e826-kserve-58445f9db7-vtnvd"
Jul 28 20:44:49.728677 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:44:49.728649    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6n4f7\" (UniqueName: \"kubernetes.io/projected/aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd-kube-api-access-6n4f7\") pod \"auth-test-model-c0e826-kserve-58445f9db7-vtnvd\" (UID: \"aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd\") " pod="ai-tenant-e2e-shared-b-c0e826/auth-test-model-c0e826-kserve-58445f9db7-vtnvd"
Jul 28 20:44:49.923894 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:44:49.923808    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c0e826/auth-test-model-c0e826-kserve-58445f9db7-vtnvd"
Jul 28 20:44:50.074986 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:44:50.074955    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c0e826/auth-test-model-c0e826-kserve-58445f9db7-vtnvd"]
Jul 28 20:44:50.079419 ip-10-0-139-54 kubenswrapper[2554]: W0728 20:44:50.079391    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaa8d2593_29e8_4e6e_a4b3_65f0bcafc0bd.slice/crio-89fa5ff19fd67a2ba3aea7c88dd62653fa74787c6c2b990ac1a685b43bd725d1 WatchSource:0}: Error finding container 89fa5ff19fd67a2ba3aea7c88dd62653fa74787c6c2b990ac1a685b43bd725d1: Status 404 returned error can't find the container with id 89fa5ff19fd67a2ba3aea7c88dd62653fa74787c6c2b990ac1a685b43bd725d1
Jul 28 20:44:50.189884 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:44:50.189792    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c0e826/auth-test-model-c0e826-kserve-58445f9db7-vtnvd" event={"ID":"aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd","Type":"ContainerStarted","Data":"3c263650ba863e0ece7948230e55f1701c6343ede2d21a3f67f883971ec986f7"}
Jul 28 20:44:50.189884 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:44:50.189836    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c0e826/auth-test-model-c0e826-kserve-58445f9db7-vtnvd" event={"ID":"aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd","Type":"ContainerStarted","Data":"89fa5ff19fd67a2ba3aea7c88dd62653fa74787c6c2b990ac1a685b43bd725d1"}
Jul 28 20:44:50.217775 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:44:50.217719    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-c0e826/auth-test-model-c0e826-kserve-58445f9db7-vtnvd" podStartSLOduration=1.217703513 podStartE2EDuration="1.217703513s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:44:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:44:50.216909779 +0000 UTC m=+2752.999921220" watchObservedRunningTime="2026-07-28 20:44:50.217703513 +0000 UTC m=+2753.000714954"
Jul 28 20:44:59.923932 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:44:59.923889    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-c0e826/auth-test-model-c0e826-kserve-58445f9db7-vtnvd"
Jul 28 20:44:59.923932 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:44:59.923941    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-c0e826/auth-test-model-c0e826-kserve-58445f9db7-vtnvd"
Jul 28 20:44:59.936591 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:44:59.936565    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-c0e826/auth-test-model-c0e826-kserve-58445f9db7-vtnvd"
Jul 28 20:45:00.247327 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:45:00.247237    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-c0e826/auth-test-model-c0e826-kserve-58445f9db7-vtnvd"
Jul 28 20:48:57.940408 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:48:57.940301    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-7vcr8_dde4c1a8-8229-4419-87b6-d8e39a0827c4/console-operator/1.log"
Jul 28 20:48:57.946919 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:48:57.946893    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-7vcr8_dde4c1a8-8229-4419-87b6-d8e39a0827c4/console-operator/1.log"
Jul 28 20:49:20.887167 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:20.887132    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c0e826/auth-test-model-c0e826-kserve-58445f9db7-vtnvd"]
Jul 28 20:49:35.906305 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:35.906267    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-c0e826/auth-test-model-c0e826-kserve-58445f9db7-vtnvd" podUID="aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd" containerName="main" containerID="cri-o://3c263650ba863e0ece7948230e55f1701c6343ede2d21a3f67f883971ec986f7" gracePeriod=45
Jul 28 20:49:36.156813 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:36.156754    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c0e826/auth-test-model-c0e826-kserve-58445f9db7-vtnvd"
Jul 28 20:49:36.286268 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:36.286239    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd-home\") pod \"aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd\" (UID: \"aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd\") "
Jul 28 20:49:36.286450 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:36.286297    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6n4f7\" (UniqueName: \"kubernetes.io/projected/aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd-kube-api-access-6n4f7\") pod \"aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd\" (UID: \"aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd\") "
Jul 28 20:49:36.286450 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:36.286345    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd-tmp-dir\") pod \"aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd\" (UID: \"aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd\") "
Jul 28 20:49:36.286450 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:36.286370    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd-model-cache\") pod \"aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd\" (UID: \"aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd\") "
Jul 28 20:49:36.286450 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:36.286396    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd-dshm\") pod \"aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd\" (UID: \"aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd\") "
Jul 28 20:49:36.286450 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:36.286443    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd-tls-certs\") pod \"aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd\" (UID: \"aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd\") "
Jul 28 20:49:36.286720 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:36.286534    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd-home" (OuterVolumeSpecName: "home") pod "aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd" (UID: "aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:49:36.286720 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:36.286641    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd" (UID: "aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:49:36.286720 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:36.286651    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd-model-cache" (OuterVolumeSpecName: "model-cache") pod "aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd" (UID: "aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:49:36.286847 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:36.286727    2554 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd-home\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:49:36.288660 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:36.288625    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd-kube-api-access-6n4f7" (OuterVolumeSpecName: "kube-api-access-6n4f7") pod "aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd" (UID: "aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd"). InnerVolumeSpecName "kube-api-access-6n4f7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:49:36.288660 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:36.288642    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd" (UID: "aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:49:36.288804 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:36.288703    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd-dshm" (OuterVolumeSpecName: "dshm") pod "aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd" (UID: "aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:49:36.388007 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:36.387970    2554 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd-model-cache\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:49:36.388007 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:36.387998    2554 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd-dshm\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:49:36.388007 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:36.388007    2554 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd-tls-certs\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:49:36.388007 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:36.388015    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6n4f7\" (UniqueName: \"kubernetes.io/projected/aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd-kube-api-access-6n4f7\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:49:36.388282 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:36.388024    2554 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd-tmp-dir\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:49:36.481011 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:36.480916    2554 generic.go:356] "Generic (PLEG): container finished" podID="aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd" containerID="3c263650ba863e0ece7948230e55f1701c6343ede2d21a3f67f883971ec986f7" exitCode=0
Jul 28 20:49:36.481011 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:36.480962    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c0e826/auth-test-model-c0e826-kserve-58445f9db7-vtnvd"
Jul 28 20:49:36.481232 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:36.481004    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c0e826/auth-test-model-c0e826-kserve-58445f9db7-vtnvd" event={"ID":"aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd","Type":"ContainerDied","Data":"3c263650ba863e0ece7948230e55f1701c6343ede2d21a3f67f883971ec986f7"}
Jul 28 20:49:36.481232 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:36.481059    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c0e826/auth-test-model-c0e826-kserve-58445f9db7-vtnvd" event={"ID":"aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd","Type":"ContainerDied","Data":"89fa5ff19fd67a2ba3aea7c88dd62653fa74787c6c2b990ac1a685b43bd725d1"}
Jul 28 20:49:36.481232 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:36.481112    2554 scope.go:117] "RemoveContainer" containerID="3c263650ba863e0ece7948230e55f1701c6343ede2d21a3f67f883971ec986f7"
Jul 28 20:49:36.490467 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:36.490448    2554 scope.go:117] "RemoveContainer" containerID="3c263650ba863e0ece7948230e55f1701c6343ede2d21a3f67f883971ec986f7"
Jul 28 20:49:36.490751 ip-10-0-139-54 kubenswrapper[2554]: E0728 20:49:36.490724    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c263650ba863e0ece7948230e55f1701c6343ede2d21a3f67f883971ec986f7\": container with ID starting with 3c263650ba863e0ece7948230e55f1701c6343ede2d21a3f67f883971ec986f7 not found: ID does not exist" containerID="3c263650ba863e0ece7948230e55f1701c6343ede2d21a3f67f883971ec986f7"
Jul 28 20:49:36.490808 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:36.490762    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c263650ba863e0ece7948230e55f1701c6343ede2d21a3f67f883971ec986f7"} err="failed to get container status \"3c263650ba863e0ece7948230e55f1701c6343ede2d21a3f67f883971ec986f7\": rpc error: code = NotFound desc = could not find container \"3c263650ba863e0ece7948230e55f1701c6343ede2d21a3f67f883971ec986f7\": container with ID starting with 3c263650ba863e0ece7948230e55f1701c6343ede2d21a3f67f883971ec986f7 not found: ID does not exist"
Jul 28 20:49:36.507250 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:36.507219    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c0e826/auth-test-model-c0e826-kserve-58445f9db7-vtnvd"]
Jul 28 20:49:36.511465 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:36.511440    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-c0e826/auth-test-model-c0e826-kserve-58445f9db7-vtnvd"]
Jul 28 20:49:37.354471 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:37.354432    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-1b6922/sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck"]
Jul 28 20:49:37.354856 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:37.354793    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd" containerName="main"
Jul 28 20:49:37.354856 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:37.354805    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd" containerName="main"
Jul 28 20:49:37.354929 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:37.354885    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd" containerName="main"
Jul 28 20:49:37.359226 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:37.359204    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1b6922/sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck"
Jul 28 20:49:37.362655 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:37.362631    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-1b6922\"/\"default-dockercfg-qz22z\""
Jul 28 20:49:37.363871 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:37.363850    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-1b6922\"/\"sub-test-model-1b6922-kserve-self-signed-certs\""
Jul 28 20:49:37.363993 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:37.363873    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-1b6922\"/\"openshift-service-ca.crt\""
Jul 28 20:49:37.363993 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:37.363892    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-1b6922\"/\"kube-root-ca.crt\""
Jul 28 20:49:37.375984 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:37.375958    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-1b6922/sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck"]
Jul 28 20:49:37.496935 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:37.496895    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9383983-1697-405a-9f08-f1192b298ac7-dshm\") pod \"sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck\" (UID: \"e9383983-1697-405a-9f08-f1192b298ac7\") " pod="ai-tenant-e2e-shared-a-1b6922/sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck"
Jul 28 20:49:37.497134 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:37.496964    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9383983-1697-405a-9f08-f1192b298ac7-home\") pod \"sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck\" (UID: \"e9383983-1697-405a-9f08-f1192b298ac7\") " pod="ai-tenant-e2e-shared-a-1b6922/sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck"
Jul 28 20:49:37.497134 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:37.497019    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9383983-1697-405a-9f08-f1192b298ac7-tmp-dir\") pod \"sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck\" (UID: \"e9383983-1697-405a-9f08-f1192b298ac7\") " pod="ai-tenant-e2e-shared-a-1b6922/sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck"
Jul 28 20:49:37.497134 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:37.497038    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gnp6\" (UniqueName: \"kubernetes.io/projected/e9383983-1697-405a-9f08-f1192b298ac7-kube-api-access-4gnp6\") pod \"sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck\" (UID: \"e9383983-1697-405a-9f08-f1192b298ac7\") " pod="ai-tenant-e2e-shared-a-1b6922/sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck"
Jul 28 20:49:37.497134 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:37.497120    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9383983-1697-405a-9f08-f1192b298ac7-tls-certs\") pod \"sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck\" (UID: \"e9383983-1697-405a-9f08-f1192b298ac7\") " pod="ai-tenant-e2e-shared-a-1b6922/sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck"
Jul 28 20:49:37.497275 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:37.497159    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9383983-1697-405a-9f08-f1192b298ac7-model-cache\") pod \"sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck\" (UID: \"e9383983-1697-405a-9f08-f1192b298ac7\") " pod="ai-tenant-e2e-shared-a-1b6922/sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck"
Jul 28 20:49:37.597950 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:37.597918    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9383983-1697-405a-9f08-f1192b298ac7-dshm\") pod \"sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck\" (UID: \"e9383983-1697-405a-9f08-f1192b298ac7\") " pod="ai-tenant-e2e-shared-a-1b6922/sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck"
Jul 28 20:49:37.598113 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:37.597976    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9383983-1697-405a-9f08-f1192b298ac7-home\") pod \"sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck\" (UID: \"e9383983-1697-405a-9f08-f1192b298ac7\") " pod="ai-tenant-e2e-shared-a-1b6922/sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck"
Jul 28 20:49:37.598113 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:37.598022    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9383983-1697-405a-9f08-f1192b298ac7-tmp-dir\") pod \"sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck\" (UID: \"e9383983-1697-405a-9f08-f1192b298ac7\") " pod="ai-tenant-e2e-shared-a-1b6922/sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck"
Jul 28 20:49:37.598113 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:37.598039    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gnp6\" (UniqueName: \"kubernetes.io/projected/e9383983-1697-405a-9f08-f1192b298ac7-kube-api-access-4gnp6\") pod \"sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck\" (UID: \"e9383983-1697-405a-9f08-f1192b298ac7\") " pod="ai-tenant-e2e-shared-a-1b6922/sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck"
Jul 28 20:49:37.598113 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:37.598066    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9383983-1697-405a-9f08-f1192b298ac7-tls-certs\") pod \"sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck\" (UID: \"e9383983-1697-405a-9f08-f1192b298ac7\") " pod="ai-tenant-e2e-shared-a-1b6922/sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck"
Jul 28 20:49:37.598324 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:37.598137    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9383983-1697-405a-9f08-f1192b298ac7-model-cache\") pod \"sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck\" (UID: \"e9383983-1697-405a-9f08-f1192b298ac7\") " pod="ai-tenant-e2e-shared-a-1b6922/sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck"
Jul 28 20:49:37.598439 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:37.598412    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9383983-1697-405a-9f08-f1192b298ac7-tmp-dir\") pod \"sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck\" (UID: \"e9383983-1697-405a-9f08-f1192b298ac7\") " pod="ai-tenant-e2e-shared-a-1b6922/sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck"
Jul 28 20:49:37.598499 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:37.598483    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9383983-1697-405a-9f08-f1192b298ac7-model-cache\") pod \"sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck\" (UID: \"e9383983-1697-405a-9f08-f1192b298ac7\") " pod="ai-tenant-e2e-shared-a-1b6922/sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck"
Jul 28 20:49:37.598499 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:37.598490    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9383983-1697-405a-9f08-f1192b298ac7-home\") pod \"sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck\" (UID: \"e9383983-1697-405a-9f08-f1192b298ac7\") " pod="ai-tenant-e2e-shared-a-1b6922/sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck"
Jul 28 20:49:37.600289 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:37.600264    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9383983-1697-405a-9f08-f1192b298ac7-dshm\") pod \"sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck\" (UID: \"e9383983-1697-405a-9f08-f1192b298ac7\") " pod="ai-tenant-e2e-shared-a-1b6922/sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck"
Jul 28 20:49:37.600502 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:37.600483    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9383983-1697-405a-9f08-f1192b298ac7-tls-certs\") pod \"sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck\" (UID: \"e9383983-1697-405a-9f08-f1192b298ac7\") " pod="ai-tenant-e2e-shared-a-1b6922/sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck"
Jul 28 20:49:37.607812 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:37.607757    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gnp6\" (UniqueName: \"kubernetes.io/projected/e9383983-1697-405a-9f08-f1192b298ac7-kube-api-access-4gnp6\") pod \"sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck\" (UID: \"e9383983-1697-405a-9f08-f1192b298ac7\") " pod="ai-tenant-e2e-shared-a-1b6922/sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck"
Jul 28 20:49:37.668494 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:37.668457    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1b6922/sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck"
Jul 28 20:49:37.778879 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:37.778839    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd" path="/var/lib/kubelet/pods/aa8d2593-29e8-4e6e-a4b3-65f0bcafc0bd/volumes"
Jul 28 20:49:37.810783 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:37.810758    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-1b6922/sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck"]
Jul 28 20:49:37.813509 ip-10-0-139-54 kubenswrapper[2554]: W0728 20:49:37.813477    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode9383983_1697_405a_9f08_f1192b298ac7.slice/crio-3062e7da8f788b148a598109a255e616c90bcd2a85aa9228e467a7b92e815ddb WatchSource:0}: Error finding container 3062e7da8f788b148a598109a255e616c90bcd2a85aa9228e467a7b92e815ddb: Status 404 returned error can't find the container with id 3062e7da8f788b148a598109a255e616c90bcd2a85aa9228e467a7b92e815ddb
Jul 28 20:49:38.493424 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:38.493389    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1b6922/sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck" event={"ID":"e9383983-1697-405a-9f08-f1192b298ac7","Type":"ContainerStarted","Data":"bbe4ada668a6f2d5793478d4ae3c99e70b5bf09b57aa61d75748efc5965f25e5"}
Jul 28 20:49:38.493424 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:38.493430    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1b6922/sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck" event={"ID":"e9383983-1697-405a-9f08-f1192b298ac7","Type":"ContainerStarted","Data":"3062e7da8f788b148a598109a255e616c90bcd2a85aa9228e467a7b92e815ddb"}
Jul 28 20:49:38.526117 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:38.524513    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-1b6922/sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck" podStartSLOduration=1.524491601 podStartE2EDuration="1.524491601s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:49:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:49:38.523575665 +0000 UTC m=+3041.306587108" watchObservedRunningTime="2026-07-28 20:49:38.524491601 +0000 UTC m=+3041.307503044"
Jul 28 20:49:47.669467 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:47.669415    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-1b6922/sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck"
Jul 28 20:49:47.669467 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:47.669482    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-1b6922/sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck"
Jul 28 20:49:47.682200 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:47.682172    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-1b6922/sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck"
Jul 28 20:49:48.551579 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:49:48.551552    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-1b6922/sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck"
Jul 28 20:51:08.261450 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:51:08.261407    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-1b6922/sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck"]
Jul 28 20:51:23.281111 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:51:23.281010    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-1b6922/sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck" podUID="e9383983-1697-405a-9f08-f1192b298ac7" containerName="main" containerID="cri-o://bbe4ada668a6f2d5793478d4ae3c99e70b5bf09b57aa61d75748efc5965f25e5" gracePeriod=45
Jul 28 20:51:23.540598 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:51:23.540572    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1b6922/sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck"
Jul 28 20:51:23.652828 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:51:23.652781    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9383983-1697-405a-9f08-f1192b298ac7-tmp-dir\") pod \"e9383983-1697-405a-9f08-f1192b298ac7\" (UID: \"e9383983-1697-405a-9f08-f1192b298ac7\") "
Jul 28 20:51:23.652828 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:51:23.652840    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9383983-1697-405a-9f08-f1192b298ac7-home\") pod \"e9383983-1697-405a-9f08-f1192b298ac7\" (UID: \"e9383983-1697-405a-9f08-f1192b298ac7\") "
Jul 28 20:51:23.653142 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:51:23.652859    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9383983-1697-405a-9f08-f1192b298ac7-dshm\") pod \"e9383983-1697-405a-9f08-f1192b298ac7\" (UID: \"e9383983-1697-405a-9f08-f1192b298ac7\") "
Jul 28 20:51:23.653142 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:51:23.652964    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4gnp6\" (UniqueName: \"kubernetes.io/projected/e9383983-1697-405a-9f08-f1192b298ac7-kube-api-access-4gnp6\") pod \"e9383983-1697-405a-9f08-f1192b298ac7\" (UID: \"e9383983-1697-405a-9f08-f1192b298ac7\") "
Jul 28 20:51:23.653142 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:51:23.653023    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9383983-1697-405a-9f08-f1192b298ac7-tls-certs\") pod \"e9383983-1697-405a-9f08-f1192b298ac7\" (UID: \"e9383983-1697-405a-9f08-f1192b298ac7\") "
Jul 28 20:51:23.653283 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:51:23.653154    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9383983-1697-405a-9f08-f1192b298ac7-model-cache\") pod \"e9383983-1697-405a-9f08-f1192b298ac7\" (UID: \"e9383983-1697-405a-9f08-f1192b298ac7\") "
Jul 28 20:51:23.653283 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:51:23.653169    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9383983-1697-405a-9f08-f1192b298ac7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e9383983-1697-405a-9f08-f1192b298ac7" (UID: "e9383983-1697-405a-9f08-f1192b298ac7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:51:23.653283 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:51:23.653195    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9383983-1697-405a-9f08-f1192b298ac7-home" (OuterVolumeSpecName: "home") pod "e9383983-1697-405a-9f08-f1192b298ac7" (UID: "e9383983-1697-405a-9f08-f1192b298ac7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:51:23.653472 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:51:23.653454    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9383983-1697-405a-9f08-f1192b298ac7-model-cache" (OuterVolumeSpecName: "model-cache") pod "e9383983-1697-405a-9f08-f1192b298ac7" (UID: "e9383983-1697-405a-9f08-f1192b298ac7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:51:23.653527 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:51:23.653479    2554 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9383983-1697-405a-9f08-f1192b298ac7-tmp-dir\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:51:23.653527 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:51:23.653499    2554 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9383983-1697-405a-9f08-f1192b298ac7-home\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:51:23.655164 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:51:23.655131    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9383983-1697-405a-9f08-f1192b298ac7-dshm" (OuterVolumeSpecName: "dshm") pod "e9383983-1697-405a-9f08-f1192b298ac7" (UID: "e9383983-1697-405a-9f08-f1192b298ac7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:51:23.655270 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:51:23.655193    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9383983-1697-405a-9f08-f1192b298ac7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e9383983-1697-405a-9f08-f1192b298ac7" (UID: "e9383983-1697-405a-9f08-f1192b298ac7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:51:23.655270 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:51:23.655231    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9383983-1697-405a-9f08-f1192b298ac7-kube-api-access-4gnp6" (OuterVolumeSpecName: "kube-api-access-4gnp6") pod "e9383983-1697-405a-9f08-f1192b298ac7" (UID: "e9383983-1697-405a-9f08-f1192b298ac7"). InnerVolumeSpecName "kube-api-access-4gnp6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:51:23.754356 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:51:23.754318    2554 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9383983-1697-405a-9f08-f1192b298ac7-dshm\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:51:23.754356 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:51:23.754347    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4gnp6\" (UniqueName: \"kubernetes.io/projected/e9383983-1697-405a-9f08-f1192b298ac7-kube-api-access-4gnp6\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:51:23.754356 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:51:23.754357    2554 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9383983-1697-405a-9f08-f1192b298ac7-tls-certs\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:51:23.754356 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:51:23.754366    2554 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9383983-1697-405a-9f08-f1192b298ac7-model-cache\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:51:23.970277 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:51:23.970178    2554 generic.go:356] "Generic (PLEG): container finished" podID="e9383983-1697-405a-9f08-f1192b298ac7" containerID="bbe4ada668a6f2d5793478d4ae3c99e70b5bf09b57aa61d75748efc5965f25e5" exitCode=0
Jul 28 20:51:23.970277 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:51:23.970222    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-1b6922/sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck"
Jul 28 20:51:23.970277 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:51:23.970262    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1b6922/sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck" event={"ID":"e9383983-1697-405a-9f08-f1192b298ac7","Type":"ContainerDied","Data":"bbe4ada668a6f2d5793478d4ae3c99e70b5bf09b57aa61d75748efc5965f25e5"}
Jul 28 20:51:23.970525 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:51:23.970304    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-1b6922/sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck" event={"ID":"e9383983-1697-405a-9f08-f1192b298ac7","Type":"ContainerDied","Data":"3062e7da8f788b148a598109a255e616c90bcd2a85aa9228e467a7b92e815ddb"}
Jul 28 20:51:23.970525 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:51:23.970322    2554 scope.go:117] "RemoveContainer" containerID="bbe4ada668a6f2d5793478d4ae3c99e70b5bf09b57aa61d75748efc5965f25e5"
Jul 28 20:51:23.979107 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:51:23.979087    2554 scope.go:117] "RemoveContainer" containerID="bbe4ada668a6f2d5793478d4ae3c99e70b5bf09b57aa61d75748efc5965f25e5"
Jul 28 20:51:23.979384 ip-10-0-139-54 kubenswrapper[2554]: E0728 20:51:23.979362    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bbe4ada668a6f2d5793478d4ae3c99e70b5bf09b57aa61d75748efc5965f25e5\": container with ID starting with bbe4ada668a6f2d5793478d4ae3c99e70b5bf09b57aa61d75748efc5965f25e5 not found: ID does not exist" containerID="bbe4ada668a6f2d5793478d4ae3c99e70b5bf09b57aa61d75748efc5965f25e5"
Jul 28 20:51:23.979447 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:51:23.979392    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbe4ada668a6f2d5793478d4ae3c99e70b5bf09b57aa61d75748efc5965f25e5"} err="failed to get container status \"bbe4ada668a6f2d5793478d4ae3c99e70b5bf09b57aa61d75748efc5965f25e5\": rpc error: code = NotFound desc = could not find container \"bbe4ada668a6f2d5793478d4ae3c99e70b5bf09b57aa61d75748efc5965f25e5\": container with ID starting with bbe4ada668a6f2d5793478d4ae3c99e70b5bf09b57aa61d75748efc5965f25e5 not found: ID does not exist"
Jul 28 20:51:23.988306 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:51:23.988279    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-1b6922/sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck"]
Jul 28 20:51:23.990521 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:51:23.990501    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-1b6922/sub-test-model-1b6922-kserve-5f47c4b8cf-kj2ck"]
Jul 28 20:51:25.777432 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:51:25.777400    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9383983-1697-405a-9f08-f1192b298ac7" path="/var/lib/kubelet/pods/e9383983-1697-405a-9f08-f1192b298ac7/volumes"
Jul 28 20:52:04.012745 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:52:04.012710    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-c0e826/rate-test-model-c0e826-kserve-f9cf84d75-j7xls"]
Jul 28 20:52:04.013238 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:52:04.013057    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9383983-1697-405a-9f08-f1192b298ac7" containerName="main"
Jul 28 20:52:04.013238 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:52:04.013082    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9383983-1697-405a-9f08-f1192b298ac7" containerName="main"
Jul 28 20:52:04.013238 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:52:04.013143    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9383983-1697-405a-9f08-f1192b298ac7" containerName="main"
Jul 28 20:52:04.017027 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:52:04.017009    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c0e826/rate-test-model-c0e826-kserve-f9cf84d75-j7xls"
Jul 28 20:52:04.021065 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:52:04.021044    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c0e826\"/\"kube-root-ca.crt\""
Jul 28 20:52:04.023136 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:52:04.023115    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c0e826\"/\"rate-test-model-c0e826-kserve-self-signed-certs\""
Jul 28 20:52:04.023251 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:52:04.023166    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c0e826\"/\"openshift-service-ca.crt\""
Jul 28 20:52:04.023314 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:52:04.023264    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c0e826\"/\"default-dockercfg-w78mt\""
Jul 28 20:52:04.039725 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:52:04.039692    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c0e826/rate-test-model-c0e826-kserve-f9cf84d75-j7xls"]
Jul 28 20:52:04.119090 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:52:04.119038    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e21e71f1-41bb-4e12-965b-ae0d9bce305e-tmp-dir\") pod \"rate-test-model-c0e826-kserve-f9cf84d75-j7xls\" (UID: \"e21e71f1-41bb-4e12-965b-ae0d9bce305e\") " pod="ai-tenant-e2e-shared-b-c0e826/rate-test-model-c0e826-kserve-f9cf84d75-j7xls"
Jul 28 20:52:04.119258 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:52:04.119096    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czs5j\" (UniqueName: \"kubernetes.io/projected/e21e71f1-41bb-4e12-965b-ae0d9bce305e-kube-api-access-czs5j\") pod \"rate-test-model-c0e826-kserve-f9cf84d75-j7xls\" (UID: \"e21e71f1-41bb-4e12-965b-ae0d9bce305e\") " pod="ai-tenant-e2e-shared-b-c0e826/rate-test-model-c0e826-kserve-f9cf84d75-j7xls"
Jul 28 20:52:04.119258 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:52:04.119135    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e21e71f1-41bb-4e12-965b-ae0d9bce305e-model-cache\") pod \"rate-test-model-c0e826-kserve-f9cf84d75-j7xls\" (UID: \"e21e71f1-41bb-4e12-965b-ae0d9bce305e\") " pod="ai-tenant-e2e-shared-b-c0e826/rate-test-model-c0e826-kserve-f9cf84d75-j7xls"
Jul 28 20:52:04.119258 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:52:04.119171    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e21e71f1-41bb-4e12-965b-ae0d9bce305e-home\") pod \"rate-test-model-c0e826-kserve-f9cf84d75-j7xls\" (UID: \"e21e71f1-41bb-4e12-965b-ae0d9bce305e\") " pod="ai-tenant-e2e-shared-b-c0e826/rate-test-model-c0e826-kserve-f9cf84d75-j7xls"
Jul 28 20:52:04.119366 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:52:04.119256    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e21e71f1-41bb-4e12-965b-ae0d9bce305e-tls-certs\") pod \"rate-test-model-c0e826-kserve-f9cf84d75-j7xls\" (UID: \"e21e71f1-41bb-4e12-965b-ae0d9bce305e\") " pod="ai-tenant-e2e-shared-b-c0e826/rate-test-model-c0e826-kserve-f9cf84d75-j7xls"
Jul 28 20:52:04.119366 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:52:04.119292    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e21e71f1-41bb-4e12-965b-ae0d9bce305e-dshm\") pod \"rate-test-model-c0e826-kserve-f9cf84d75-j7xls\" (UID: \"e21e71f1-41bb-4e12-965b-ae0d9bce305e\") " pod="ai-tenant-e2e-shared-b-c0e826/rate-test-model-c0e826-kserve-f9cf84d75-j7xls"
Jul 28 20:52:04.220127 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:52:04.220089    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e21e71f1-41bb-4e12-965b-ae0d9bce305e-dshm\") pod \"rate-test-model-c0e826-kserve-f9cf84d75-j7xls\" (UID: \"e21e71f1-41bb-4e12-965b-ae0d9bce305e\") " pod="ai-tenant-e2e-shared-b-c0e826/rate-test-model-c0e826-kserve-f9cf84d75-j7xls"
Jul 28 20:52:04.220312 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:52:04.220167    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e21e71f1-41bb-4e12-965b-ae0d9bce305e-tmp-dir\") pod \"rate-test-model-c0e826-kserve-f9cf84d75-j7xls\" (UID: \"e21e71f1-41bb-4e12-965b-ae0d9bce305e\") " pod="ai-tenant-e2e-shared-b-c0e826/rate-test-model-c0e826-kserve-f9cf84d75-j7xls"
Jul 28 20:52:04.220312 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:52:04.220191    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czs5j\" (UniqueName: \"kubernetes.io/projected/e21e71f1-41bb-4e12-965b-ae0d9bce305e-kube-api-access-czs5j\") pod \"rate-test-model-c0e826-kserve-f9cf84d75-j7xls\" (UID: \"e21e71f1-41bb-4e12-965b-ae0d9bce305e\") " pod="ai-tenant-e2e-shared-b-c0e826/rate-test-model-c0e826-kserve-f9cf84d75-j7xls"
Jul 28 20:52:04.220312 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:52:04.220213    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e21e71f1-41bb-4e12-965b-ae0d9bce305e-model-cache\") pod \"rate-test-model-c0e826-kserve-f9cf84d75-j7xls\" (UID: \"e21e71f1-41bb-4e12-965b-ae0d9bce305e\") " pod="ai-tenant-e2e-shared-b-c0e826/rate-test-model-c0e826-kserve-f9cf84d75-j7xls"
Jul 28 20:52:04.220312 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:52:04.220264    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e21e71f1-41bb-4e12-965b-ae0d9bce305e-home\") pod \"rate-test-model-c0e826-kserve-f9cf84d75-j7xls\" (UID: \"e21e71f1-41bb-4e12-965b-ae0d9bce305e\") " pod="ai-tenant-e2e-shared-b-c0e826/rate-test-model-c0e826-kserve-f9cf84d75-j7xls"
Jul 28 20:52:04.220512 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:52:04.220314    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e21e71f1-41bb-4e12-965b-ae0d9bce305e-tls-certs\") pod \"rate-test-model-c0e826-kserve-f9cf84d75-j7xls\" (UID: \"e21e71f1-41bb-4e12-965b-ae0d9bce305e\") " pod="ai-tenant-e2e-shared-b-c0e826/rate-test-model-c0e826-kserve-f9cf84d75-j7xls"
Jul 28 20:52:04.220668 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:52:04.220640    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e21e71f1-41bb-4e12-965b-ae0d9bce305e-model-cache\") pod \"rate-test-model-c0e826-kserve-f9cf84d75-j7xls\" (UID: \"e21e71f1-41bb-4e12-965b-ae0d9bce305e\") " pod="ai-tenant-e2e-shared-b-c0e826/rate-test-model-c0e826-kserve-f9cf84d75-j7xls"
Jul 28 20:52:04.220772 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:52:04.220665    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e21e71f1-41bb-4e12-965b-ae0d9bce305e-tmp-dir\") pod \"rate-test-model-c0e826-kserve-f9cf84d75-j7xls\" (UID: \"e21e71f1-41bb-4e12-965b-ae0d9bce305e\") " pod="ai-tenant-e2e-shared-b-c0e826/rate-test-model-c0e826-kserve-f9cf84d75-j7xls"
Jul 28 20:52:04.220772 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:52:04.220720    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e21e71f1-41bb-4e12-965b-ae0d9bce305e-home\") pod \"rate-test-model-c0e826-kserve-f9cf84d75-j7xls\" (UID: \"e21e71f1-41bb-4e12-965b-ae0d9bce305e\") " pod="ai-tenant-e2e-shared-b-c0e826/rate-test-model-c0e826-kserve-f9cf84d75-j7xls"
Jul 28 20:52:04.222687 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:52:04.222662    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e21e71f1-41bb-4e12-965b-ae0d9bce305e-dshm\") pod \"rate-test-model-c0e826-kserve-f9cf84d75-j7xls\" (UID: \"e21e71f1-41bb-4e12-965b-ae0d9bce305e\") " pod="ai-tenant-e2e-shared-b-c0e826/rate-test-model-c0e826-kserve-f9cf84d75-j7xls"
Jul 28 20:52:04.222687 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:52:04.222684    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e21e71f1-41bb-4e12-965b-ae0d9bce305e-tls-certs\") pod \"rate-test-model-c0e826-kserve-f9cf84d75-j7xls\" (UID: \"e21e71f1-41bb-4e12-965b-ae0d9bce305e\") " pod="ai-tenant-e2e-shared-b-c0e826/rate-test-model-c0e826-kserve-f9cf84d75-j7xls"
Jul 28 20:52:04.235955 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:52:04.235935    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czs5j\" (UniqueName: \"kubernetes.io/projected/e21e71f1-41bb-4e12-965b-ae0d9bce305e-kube-api-access-czs5j\") pod \"rate-test-model-c0e826-kserve-f9cf84d75-j7xls\" (UID: \"e21e71f1-41bb-4e12-965b-ae0d9bce305e\") " pod="ai-tenant-e2e-shared-b-c0e826/rate-test-model-c0e826-kserve-f9cf84d75-j7xls"
Jul 28 20:52:04.326508 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:52:04.326474    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c0e826/rate-test-model-c0e826-kserve-f9cf84d75-j7xls"
Jul 28 20:52:04.464363 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:52:04.464338    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c0e826/rate-test-model-c0e826-kserve-f9cf84d75-j7xls"]
Jul 28 20:52:04.465355 ip-10-0-139-54 kubenswrapper[2554]: W0728 20:52:04.465333    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode21e71f1_41bb_4e12_965b_ae0d9bce305e.slice/crio-44cc08205a6c191235b6a29e00950ba5895ddf72ff272ca2ba0221658eb1893e WatchSource:0}: Error finding container 44cc08205a6c191235b6a29e00950ba5895ddf72ff272ca2ba0221658eb1893e: Status 404 returned error can't find the container with id 44cc08205a6c191235b6a29e00950ba5895ddf72ff272ca2ba0221658eb1893e
Jul 28 20:52:05.162234 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:52:05.162193    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c0e826/rate-test-model-c0e826-kserve-f9cf84d75-j7xls" event={"ID":"e21e71f1-41bb-4e12-965b-ae0d9bce305e","Type":"ContainerStarted","Data":"f4121eb1bbaf95e240c53565b75d4a31c5dfa809fc1f47c5c82aefab1c775c39"}
Jul 28 20:52:05.162234 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:52:05.162240    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c0e826/rate-test-model-c0e826-kserve-f9cf84d75-j7xls" event={"ID":"e21e71f1-41bb-4e12-965b-ae0d9bce305e","Type":"ContainerStarted","Data":"44cc08205a6c191235b6a29e00950ba5895ddf72ff272ca2ba0221658eb1893e"}
Jul 28 20:52:14.327023 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:52:14.326983    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-c0e826/rate-test-model-c0e826-kserve-f9cf84d75-j7xls"
Jul 28 20:52:14.327621 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:52:14.327138    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-c0e826/rate-test-model-c0e826-kserve-f9cf84d75-j7xls"
Jul 28 20:52:14.339468 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:52:14.339442    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-c0e826/rate-test-model-c0e826-kserve-f9cf84d75-j7xls"
Jul 28 20:52:14.358286 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:52:14.358237    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-c0e826/rate-test-model-c0e826-kserve-f9cf84d75-j7xls" podStartSLOduration=11.358221427 podStartE2EDuration="11.358221427s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:52:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:52:05.205859695 +0000 UTC m=+3187.988871137" watchObservedRunningTime="2026-07-28 20:52:14.358221427 +0000 UTC m=+3197.141232868"
Jul 28 20:52:15.225753 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:52:15.225718    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-c0e826/rate-test-model-c0e826-kserve-f9cf84d75-j7xls"
Jul 28 20:53:06.595414 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:06.595326    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c0e826/rate-test-model-c0e826-kserve-f9cf84d75-j7xls"]
Jul 28 20:53:07.634579 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:07.634544    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7507-139960181915056"]
Jul 28 20:53:07.638058 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:07.638041    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7507-139960181915056"
Jul 28 20:53:07.641124 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:07.641099    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-26m6p\""
Jul 28 20:53:07.646724 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:07.646701    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7507-139960181915056"]
Jul 28 20:53:07.704675 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:07.704639    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94cnt\" (UniqueName: \"kubernetes.io/projected/83a7a94f-ba99-4833-a741-b76bedb02980-kube-api-access-94cnt\") pod \"test-curl-7507-139960181915056\" (UID: \"83a7a94f-ba99-4833-a741-b76bedb02980\") " pod="opendatahub/test-curl-7507-139960181915056"
Jul 28 20:53:07.805707 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:07.805673    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94cnt\" (UniqueName: \"kubernetes.io/projected/83a7a94f-ba99-4833-a741-b76bedb02980-kube-api-access-94cnt\") pod \"test-curl-7507-139960181915056\" (UID: \"83a7a94f-ba99-4833-a741-b76bedb02980\") " pod="opendatahub/test-curl-7507-139960181915056"
Jul 28 20:53:07.815664 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:07.815632    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94cnt\" (UniqueName: \"kubernetes.io/projected/83a7a94f-ba99-4833-a741-b76bedb02980-kube-api-access-94cnt\") pod \"test-curl-7507-139960181915056\" (UID: \"83a7a94f-ba99-4833-a741-b76bedb02980\") " pod="opendatahub/test-curl-7507-139960181915056"
Jul 28 20:53:07.947580 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:07.947497    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7507-139960181915056"
Jul 28 20:53:08.080659 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:08.080629    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7507-139960181915056"]
Jul 28 20:53:08.083872 ip-10-0-139-54 kubenswrapper[2554]: W0728 20:53:08.083841    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod83a7a94f_ba99_4833_a741_b76bedb02980.slice/crio-21f663d8ce45c1bc9f9912ee969887648fc4fe25b3de22458b457673cab589d7 WatchSource:0}: Error finding container 21f663d8ce45c1bc9f9912ee969887648fc4fe25b3de22458b457673cab589d7: Status 404 returned error can't find the container with id 21f663d8ce45c1bc9f9912ee969887648fc4fe25b3de22458b457673cab589d7
Jul 28 20:53:08.085525 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:08.085508    2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 20:53:08.468755 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:08.468716    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7507-139960181915056" event={"ID":"83a7a94f-ba99-4833-a741-b76bedb02980","Type":"ContainerStarted","Data":"21f663d8ce45c1bc9f9912ee969887648fc4fe25b3de22458b457673cab589d7"}
Jul 28 20:53:09.475401 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:09.475365    2554 generic.go:356] "Generic (PLEG): container finished" podID="83a7a94f-ba99-4833-a741-b76bedb02980" containerID="3a68ad79ecc9656e18335deb83eabb4bd6175450f07940c612155e956302eb25" exitCode=0
Jul 28 20:53:09.475862 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:09.475406    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7507-139960181915056" event={"ID":"83a7a94f-ba99-4833-a741-b76bedb02980","Type":"ContainerDied","Data":"3a68ad79ecc9656e18335deb83eabb4bd6175450f07940c612155e956302eb25"}
Jul 28 20:53:10.612783 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:10.612761    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7507-139960181915056"
Jul 28 20:53:10.650526 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:10.650497    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7507-139960181915056_83a7a94f-ba99-4833-a741-b76bedb02980/test-curl-7507-139960181915056/0.log"
Jul 28 20:53:10.672560 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:10.672536    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7507-139960181915056"]
Jul 28 20:53:10.679096 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:10.678201    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7507-139960181915056"]
Jul 28 20:53:10.730184 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:10.730157    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-94cnt\" (UniqueName: \"kubernetes.io/projected/83a7a94f-ba99-4833-a741-b76bedb02980-kube-api-access-94cnt\") pod \"83a7a94f-ba99-4833-a741-b76bedb02980\" (UID: \"83a7a94f-ba99-4833-a741-b76bedb02980\") "
Jul 28 20:53:10.732408 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:10.732345    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83a7a94f-ba99-4833-a741-b76bedb02980-kube-api-access-94cnt" (OuterVolumeSpecName: "kube-api-access-94cnt") pod "83a7a94f-ba99-4833-a741-b76bedb02980" (UID: "83a7a94f-ba99-4833-a741-b76bedb02980"). InnerVolumeSpecName "kube-api-access-94cnt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:53:10.798739 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:10.798708    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7507-139960182006576"]
Jul 28 20:53:10.799114 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:10.799100    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83a7a94f-ba99-4833-a741-b76bedb02980" containerName="test-curl-7507-139960181915056"
Jul 28 20:53:10.799173 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:10.799116    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="83a7a94f-ba99-4833-a741-b76bedb02980" containerName="test-curl-7507-139960181915056"
Jul 28 20:53:10.799216 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:10.799207    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="83a7a94f-ba99-4833-a741-b76bedb02980" containerName="test-curl-7507-139960181915056"
Jul 28 20:53:10.802175 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:10.802158    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7507-139960182006576"
Jul 28 20:53:10.809931 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:10.809904    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7507-139960182006576"]
Jul 28 20:53:10.831132 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:10.831100    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-94cnt\" (UniqueName: \"kubernetes.io/projected/83a7a94f-ba99-4833-a741-b76bedb02980-kube-api-access-94cnt\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:53:10.932425 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:10.932382    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58ft6\" (UniqueName: \"kubernetes.io/projected/d391bd13-0a34-420f-9ac5-271ecb50c64b-kube-api-access-58ft6\") pod \"test-curl-7507-139960182006576\" (UID: \"d391bd13-0a34-420f-9ac5-271ecb50c64b\") " pod="opendatahub/test-curl-7507-139960182006576"
Jul 28 20:53:11.033734 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:11.033646    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58ft6\" (UniqueName: \"kubernetes.io/projected/d391bd13-0a34-420f-9ac5-271ecb50c64b-kube-api-access-58ft6\") pod \"test-curl-7507-139960182006576\" (UID: \"d391bd13-0a34-420f-9ac5-271ecb50c64b\") " pod="opendatahub/test-curl-7507-139960182006576"
Jul 28 20:53:11.042420 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:11.042390    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58ft6\" (UniqueName: \"kubernetes.io/projected/d391bd13-0a34-420f-9ac5-271ecb50c64b-kube-api-access-58ft6\") pod \"test-curl-7507-139960182006576\" (UID: \"d391bd13-0a34-420f-9ac5-271ecb50c64b\") " pod="opendatahub/test-curl-7507-139960182006576"
Jul 28 20:53:11.112386 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:11.112349    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7507-139960182006576"
Jul 28 20:53:11.242865 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:11.242842    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7507-139960182006576"]
Jul 28 20:53:11.244747 ip-10-0-139-54 kubenswrapper[2554]: W0728 20:53:11.244720    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd391bd13_0a34_420f_9ac5_271ecb50c64b.slice/crio-1aeeea51b20207b3199ccb2d90e7bf8de99e95708d8476b31332d85aa9ef22a1 WatchSource:0}: Error finding container 1aeeea51b20207b3199ccb2d90e7bf8de99e95708d8476b31332d85aa9ef22a1: Status 404 returned error can't find the container with id 1aeeea51b20207b3199ccb2d90e7bf8de99e95708d8476b31332d85aa9ef22a1
Jul 28 20:53:11.485761 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:11.485730    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7507-139960181915056"
Jul 28 20:53:11.485921 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:11.485729    2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="21f663d8ce45c1bc9f9912ee969887648fc4fe25b3de22458b457673cab589d7"
Jul 28 20:53:11.486884 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:11.486859    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7507-139960182006576" event={"ID":"d391bd13-0a34-420f-9ac5-271ecb50c64b","Type":"ContainerStarted","Data":"1aeeea51b20207b3199ccb2d90e7bf8de99e95708d8476b31332d85aa9ef22a1"}
Jul 28 20:53:11.497457 ip-10-0-139-54 kubenswrapper[2554]: E0728 20:53:11.497431    2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7507-139960181915056\" is forbidden: User \"system:node:ip-10-0-139-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-54.ec2.internal' and this object" podUID="83a7a94f-ba99-4833-a741-b76bedb02980" pod="opendatahub/test-curl-7507-139960181915056"
Jul 28 20:53:11.777488 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:11.777412    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83a7a94f-ba99-4833-a741-b76bedb02980" path="/var/lib/kubelet/pods/83a7a94f-ba99-4833-a741-b76bedb02980/volumes"
Jul 28 20:53:12.492361 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:12.492320    2554 generic.go:356] "Generic (PLEG): container finished" podID="d391bd13-0a34-420f-9ac5-271ecb50c64b" containerID="5d1d053b83120f4f245b3d567852cc2d36a1fabde25b4f001978ee60f0a75117" exitCode=0
Jul 28 20:53:12.492555 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:12.492383    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7507-139960182006576" event={"ID":"d391bd13-0a34-420f-9ac5-271ecb50c64b","Type":"ContainerDied","Data":"5d1d053b83120f4f245b3d567852cc2d36a1fabde25b4f001978ee60f0a75117"}
Jul 28 20:53:13.630865 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:13.630842    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7507-139960182006576"
Jul 28 20:53:13.658902 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:13.658873    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7507-139960182006576_d391bd13-0a34-420f-9ac5-271ecb50c64b/test-curl-7507-139960182006576/0.log"
Jul 28 20:53:13.683811 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:13.683772    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7507-139960182006576"]
Jul 28 20:53:13.685432 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:13.685408    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7507-139960182006576"]
Jul 28 20:53:13.758995 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:13.758904    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-58ft6\" (UniqueName: \"kubernetes.io/projected/d391bd13-0a34-420f-9ac5-271ecb50c64b-kube-api-access-58ft6\") pod \"d391bd13-0a34-420f-9ac5-271ecb50c64b\" (UID: \"d391bd13-0a34-420f-9ac5-271ecb50c64b\") "
Jul 28 20:53:13.761042 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:13.761016    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d391bd13-0a34-420f-9ac5-271ecb50c64b-kube-api-access-58ft6" (OuterVolumeSpecName: "kube-api-access-58ft6") pod "d391bd13-0a34-420f-9ac5-271ecb50c64b" (UID: "d391bd13-0a34-420f-9ac5-271ecb50c64b"). InnerVolumeSpecName "kube-api-access-58ft6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:53:13.777888 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:13.777858    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d391bd13-0a34-420f-9ac5-271ecb50c64b" path="/var/lib/kubelet/pods/d391bd13-0a34-420f-9ac5-271ecb50c64b/volumes"
Jul 28 20:53:13.848420 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:13.848388    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7507-139960181991440"]
Jul 28 20:53:13.848743 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:13.848731    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d391bd13-0a34-420f-9ac5-271ecb50c64b" containerName="test-curl-7507-139960182006576"
Jul 28 20:53:13.848786 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:13.848745    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="d391bd13-0a34-420f-9ac5-271ecb50c64b" containerName="test-curl-7507-139960182006576"
Jul 28 20:53:13.848824 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:13.848811    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="d391bd13-0a34-420f-9ac5-271ecb50c64b" containerName="test-curl-7507-139960182006576"
Jul 28 20:53:13.851735 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:13.851719    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7507-139960181991440"
Jul 28 20:53:13.860386 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:13.860363    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7507-139960181991440"]
Jul 28 20:53:13.860641 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:13.860621    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-58ft6\" (UniqueName: \"kubernetes.io/projected/d391bd13-0a34-420f-9ac5-271ecb50c64b-kube-api-access-58ft6\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:53:13.961212 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:13.961173    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqljd\" (UniqueName: \"kubernetes.io/projected/4f7e98b4-63ce-454d-98c1-784002f219f1-kube-api-access-fqljd\") pod \"test-curl-7507-139960181991440\" (UID: \"4f7e98b4-63ce-454d-98c1-784002f219f1\") " pod="opendatahub/test-curl-7507-139960181991440"
Jul 28 20:53:14.062099 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:14.062028    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqljd\" (UniqueName: \"kubernetes.io/projected/4f7e98b4-63ce-454d-98c1-784002f219f1-kube-api-access-fqljd\") pod \"test-curl-7507-139960181991440\" (UID: \"4f7e98b4-63ce-454d-98c1-784002f219f1\") " pod="opendatahub/test-curl-7507-139960181991440"
Jul 28 20:53:14.074962 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:14.074937    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqljd\" (UniqueName: \"kubernetes.io/projected/4f7e98b4-63ce-454d-98c1-784002f219f1-kube-api-access-fqljd\") pod \"test-curl-7507-139960181991440\" (UID: \"4f7e98b4-63ce-454d-98c1-784002f219f1\") " pod="opendatahub/test-curl-7507-139960181991440"
Jul 28 20:53:14.161475 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:14.161440    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7507-139960181991440"
Jul 28 20:53:14.293259 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:14.293228    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7507-139960181991440"]
Jul 28 20:53:14.294702 ip-10-0-139-54 kubenswrapper[2554]: W0728 20:53:14.294676    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4f7e98b4_63ce_454d_98c1_784002f219f1.slice/crio-feeff3ee723e724f1af7b0b94a816536500e852d562aff9146eac30948937127 WatchSource:0}: Error finding container feeff3ee723e724f1af7b0b94a816536500e852d562aff9146eac30948937127: Status 404 returned error can't find the container with id feeff3ee723e724f1af7b0b94a816536500e852d562aff9146eac30948937127
Jul 28 20:53:14.502900 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:14.502862    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7507-139960181991440" event={"ID":"4f7e98b4-63ce-454d-98c1-784002f219f1","Type":"ContainerStarted","Data":"feeff3ee723e724f1af7b0b94a816536500e852d562aff9146eac30948937127"}
Jul 28 20:53:14.504110 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:14.504088    2554 scope.go:117] "RemoveContainer" containerID="5d1d053b83120f4f245b3d567852cc2d36a1fabde25b4f001978ee60f0a75117"
Jul 28 20:53:14.504110 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:14.504099    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7507-139960182006576"
Jul 28 20:53:15.510437 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:15.510404    2554 generic.go:356] "Generic (PLEG): container finished" podID="4f7e98b4-63ce-454d-98c1-784002f219f1" containerID="23b931796725d5e20981b1f4493697f01d33ae1793ab9ce56cf01148ed5f799c" exitCode=0
Jul 28 20:53:15.510878 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:15.510495    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7507-139960181991440" event={"ID":"4f7e98b4-63ce-454d-98c1-784002f219f1","Type":"ContainerDied","Data":"23b931796725d5e20981b1f4493697f01d33ae1793ab9ce56cf01148ed5f799c"}
Jul 28 20:53:16.647311 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:16.647283    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7507-139960181991440"
Jul 28 20:53:16.669670 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:16.669644    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7507-139960181991440_4f7e98b4-63ce-454d-98c1-784002f219f1/test-curl-7507-139960181991440/0.log"
Jul 28 20:53:16.694665 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:16.694631    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7507-139960181991440"]
Jul 28 20:53:16.699396 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:16.699358    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7507-139960181991440"]
Jul 28 20:53:16.786062 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:16.786037    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fqljd\" (UniqueName: \"kubernetes.io/projected/4f7e98b4-63ce-454d-98c1-784002f219f1-kube-api-access-fqljd\") pod \"4f7e98b4-63ce-454d-98c1-784002f219f1\" (UID: \"4f7e98b4-63ce-454d-98c1-784002f219f1\") "
Jul 28 20:53:16.788111 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:16.788086    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f7e98b4-63ce-454d-98c1-784002f219f1-kube-api-access-fqljd" (OuterVolumeSpecName: "kube-api-access-fqljd") pod "4f7e98b4-63ce-454d-98c1-784002f219f1" (UID: "4f7e98b4-63ce-454d-98c1-784002f219f1"). InnerVolumeSpecName "kube-api-access-fqljd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:53:16.830694 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:16.830661    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7507-139960181991440"]
Jul 28 20:53:16.831109 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:16.831089    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f7e98b4-63ce-454d-98c1-784002f219f1" containerName="test-curl-7507-139960181991440"
Jul 28 20:53:16.831205 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:16.831111    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f7e98b4-63ce-454d-98c1-784002f219f1" containerName="test-curl-7507-139960181991440"
Jul 28 20:53:16.831260 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:16.831236    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f7e98b4-63ce-454d-98c1-784002f219f1" containerName="test-curl-7507-139960181991440"
Jul 28 20:53:16.834204 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:16.834185    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7507-139960181991440"
Jul 28 20:53:16.840765 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:16.840742    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7507-139960181991440"]
Jul 28 20:53:16.886852 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:16.886815    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fqljd\" (UniqueName: \"kubernetes.io/projected/4f7e98b4-63ce-454d-98c1-784002f219f1-kube-api-access-fqljd\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:53:16.987647 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:16.987609    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kb965\" (UniqueName: \"kubernetes.io/projected/897af9ce-3d32-4c7b-a112-a6da1469993f-kube-api-access-kb965\") pod \"test-curl-7507-139960181991440\" (UID: \"897af9ce-3d32-4c7b-a112-a6da1469993f\") " pod="opendatahub/test-curl-7507-139960181991440"
Jul 28 20:53:17.089030 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:17.088937    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kb965\" (UniqueName: \"kubernetes.io/projected/897af9ce-3d32-4c7b-a112-a6da1469993f-kube-api-access-kb965\") pod \"test-curl-7507-139960181991440\" (UID: \"897af9ce-3d32-4c7b-a112-a6da1469993f\") " pod="opendatahub/test-curl-7507-139960181991440"
Jul 28 20:53:17.098659 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:17.098591    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kb965\" (UniqueName: \"kubernetes.io/projected/897af9ce-3d32-4c7b-a112-a6da1469993f-kube-api-access-kb965\") pod \"test-curl-7507-139960181991440\" (UID: \"897af9ce-3d32-4c7b-a112-a6da1469993f\") " pod="opendatahub/test-curl-7507-139960181991440"
Jul 28 20:53:17.144406 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:17.144371    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7507-139960181991440"
Jul 28 20:53:17.281188 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:17.281157    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7507-139960181991440"]
Jul 28 20:53:17.284864 ip-10-0-139-54 kubenswrapper[2554]: W0728 20:53:17.284836    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod897af9ce_3d32_4c7b_a112_a6da1469993f.slice/crio-d91500ac71b385ba12492a4f1800064465a9a8276f0ed6793d71f574ebceb860 WatchSource:0}: Error finding container d91500ac71b385ba12492a4f1800064465a9a8276f0ed6793d71f574ebceb860: Status 404 returned error can't find the container with id d91500ac71b385ba12492a4f1800064465a9a8276f0ed6793d71f574ebceb860
Jul 28 20:53:17.522590 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:17.522509    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7507-139960181991440"
Jul 28 20:53:17.522732 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:17.522507    2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="feeff3ee723e724f1af7b0b94a816536500e852d562aff9146eac30948937127"
Jul 28 20:53:17.523788 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:17.523762    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7507-139960181991440" event={"ID":"897af9ce-3d32-4c7b-a112-a6da1469993f","Type":"ContainerStarted","Data":"d91500ac71b385ba12492a4f1800064465a9a8276f0ed6793d71f574ebceb860"}
Jul 28 20:53:17.535133 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:17.535103    2554 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7507-139960181991440" oldPodUID="4f7e98b4-63ce-454d-98c1-784002f219f1" podUID="897af9ce-3d32-4c7b-a112-a6da1469993f"
Jul 28 20:53:17.778160 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:17.778089    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f7e98b4-63ce-454d-98c1-784002f219f1" path="/var/lib/kubelet/pods/4f7e98b4-63ce-454d-98c1-784002f219f1/volumes"
Jul 28 20:53:18.529873 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:18.529840    2554 generic.go:356] "Generic (PLEG): container finished" podID="897af9ce-3d32-4c7b-a112-a6da1469993f" containerID="56a83a8545fb7b541b850906a2dbf1d7a9cd7956697d4e88608466ea009e152f" exitCode=0
Jul 28 20:53:18.530046 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:18.529885    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7507-139960181991440" event={"ID":"897af9ce-3d32-4c7b-a112-a6da1469993f","Type":"ContainerDied","Data":"56a83a8545fb7b541b850906a2dbf1d7a9cd7956697d4e88608466ea009e152f"}
Jul 28 20:53:19.665510 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:19.665486    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7507-139960181991440"
Jul 28 20:53:19.686867 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:19.686841    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7507-139960181991440_897af9ce-3d32-4c7b-a112-a6da1469993f/test-curl-7507-139960181991440/0.log"
Jul 28 20:53:19.709474 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:19.709448    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7507-139960181991440"]
Jul 28 20:53:19.713723 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:19.713696    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7507-139960181991440"]
Jul 28 20:53:19.811921 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:19.811888    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kb965\" (UniqueName: \"kubernetes.io/projected/897af9ce-3d32-4c7b-a112-a6da1469993f-kube-api-access-kb965\") pod \"897af9ce-3d32-4c7b-a112-a6da1469993f\" (UID: \"897af9ce-3d32-4c7b-a112-a6da1469993f\") "
Jul 28 20:53:19.813927 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:19.813904    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/897af9ce-3d32-4c7b-a112-a6da1469993f-kube-api-access-kb965" (OuterVolumeSpecName: "kube-api-access-kb965") pod "897af9ce-3d32-4c7b-a112-a6da1469993f" (UID: "897af9ce-3d32-4c7b-a112-a6da1469993f"). InnerVolumeSpecName "kube-api-access-kb965". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:53:19.848925 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:19.848892    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7507-139960181992448"]
Jul 28 20:53:19.849247 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:19.849235    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="897af9ce-3d32-4c7b-a112-a6da1469993f" containerName="test-curl-7507-139960181991440"
Jul 28 20:53:19.849291 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:19.849249    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="897af9ce-3d32-4c7b-a112-a6da1469993f" containerName="test-curl-7507-139960181991440"
Jul 28 20:53:19.849332 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:19.849312    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="897af9ce-3d32-4c7b-a112-a6da1469993f" containerName="test-curl-7507-139960181991440"
Jul 28 20:53:19.852095 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:19.852063    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7507-139960181992448"
Jul 28 20:53:19.859318 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:19.859171    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7507-139960181992448"]
Jul 28 20:53:19.912699 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:19.912659    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kb965\" (UniqueName: \"kubernetes.io/projected/897af9ce-3d32-4c7b-a112-a6da1469993f-kube-api-access-kb965\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:53:20.013668 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:20.013625    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gp66n\" (UniqueName: \"kubernetes.io/projected/1f072f26-1a86-4134-8e65-7c53d77d74e4-kube-api-access-gp66n\") pod \"test-curl-7507-139960181992448\" (UID: \"1f072f26-1a86-4134-8e65-7c53d77d74e4\") " pod="opendatahub/test-curl-7507-139960181992448"
Jul 28 20:53:20.114532 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:20.114434    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gp66n\" (UniqueName: \"kubernetes.io/projected/1f072f26-1a86-4134-8e65-7c53d77d74e4-kube-api-access-gp66n\") pod \"test-curl-7507-139960181992448\" (UID: \"1f072f26-1a86-4134-8e65-7c53d77d74e4\") " pod="opendatahub/test-curl-7507-139960181992448"
Jul 28 20:53:20.124421 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:20.124386    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gp66n\" (UniqueName: \"kubernetes.io/projected/1f072f26-1a86-4134-8e65-7c53d77d74e4-kube-api-access-gp66n\") pod \"test-curl-7507-139960181992448\" (UID: \"1f072f26-1a86-4134-8e65-7c53d77d74e4\") " pod="opendatahub/test-curl-7507-139960181992448"
Jul 28 20:53:20.161550 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:20.161510    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7507-139960181992448"
Jul 28 20:53:20.290921 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:20.290893    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7507-139960181992448"]
Jul 28 20:53:20.293336 ip-10-0-139-54 kubenswrapper[2554]: W0728 20:53:20.293311    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1f072f26_1a86_4134_8e65_7c53d77d74e4.slice/crio-e11cc934958611be4bee7cc73f7f77b9071d705c632791bda7faa7066120c3bd WatchSource:0}: Error finding container e11cc934958611be4bee7cc73f7f77b9071d705c632791bda7faa7066120c3bd: Status 404 returned error can't find the container with id e11cc934958611be4bee7cc73f7f77b9071d705c632791bda7faa7066120c3bd
Jul 28 20:53:20.539352 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:20.539322    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7507-139960181992448" event={"ID":"1f072f26-1a86-4134-8e65-7c53d77d74e4","Type":"ContainerStarted","Data":"e11cc934958611be4bee7cc73f7f77b9071d705c632791bda7faa7066120c3bd"}
Jul 28 20:53:20.540614 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:20.540592    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7507-139960181991440"
Jul 28 20:53:20.540715 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:20.540598    2554 scope.go:117] "RemoveContainer" containerID="56a83a8545fb7b541b850906a2dbf1d7a9cd7956697d4e88608466ea009e152f"
Jul 28 20:53:21.547487 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:21.547447    2554 generic.go:356] "Generic (PLEG): container finished" podID="1f072f26-1a86-4134-8e65-7c53d77d74e4" containerID="c7b7a0956eb7b24ab10e936dfdd48b634501746b1c79052262e9f0f33d94141b" exitCode=0
Jul 28 20:53:21.547888 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:21.547539    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7507-139960181992448" event={"ID":"1f072f26-1a86-4134-8e65-7c53d77d74e4","Type":"ContainerDied","Data":"c7b7a0956eb7b24ab10e936dfdd48b634501746b1c79052262e9f0f33d94141b"}
Jul 28 20:53:21.614274 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:21.614230    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-c0e826/rate-test-model-c0e826-kserve-f9cf84d75-j7xls" podUID="e21e71f1-41bb-4e12-965b-ae0d9bce305e" containerName="main" containerID="cri-o://f4121eb1bbaf95e240c53565b75d4a31c5dfa809fc1f47c5c82aefab1c775c39" gracePeriod=45
Jul 28 20:53:21.778764 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:21.778728    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="897af9ce-3d32-4c7b-a112-a6da1469993f" path="/var/lib/kubelet/pods/897af9ce-3d32-4c7b-a112-a6da1469993f/volumes"
Jul 28 20:53:21.865875 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:21.865843    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c0e826/rate-test-model-c0e826-kserve-f9cf84d75-j7xls"
Jul 28 20:53:22.032883 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:22.032843    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e21e71f1-41bb-4e12-965b-ae0d9bce305e-dshm\") pod \"e21e71f1-41bb-4e12-965b-ae0d9bce305e\" (UID: \"e21e71f1-41bb-4e12-965b-ae0d9bce305e\") "
Jul 28 20:53:22.033334 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:22.032895    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e21e71f1-41bb-4e12-965b-ae0d9bce305e-tls-certs\") pod \"e21e71f1-41bb-4e12-965b-ae0d9bce305e\" (UID: \"e21e71f1-41bb-4e12-965b-ae0d9bce305e\") "
Jul 28 20:53:22.033334 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:22.032946    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-czs5j\" (UniqueName: \"kubernetes.io/projected/e21e71f1-41bb-4e12-965b-ae0d9bce305e-kube-api-access-czs5j\") pod \"e21e71f1-41bb-4e12-965b-ae0d9bce305e\" (UID: \"e21e71f1-41bb-4e12-965b-ae0d9bce305e\") "
Jul 28 20:53:22.033334 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:22.032965    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e21e71f1-41bb-4e12-965b-ae0d9bce305e-home\") pod \"e21e71f1-41bb-4e12-965b-ae0d9bce305e\" (UID: \"e21e71f1-41bb-4e12-965b-ae0d9bce305e\") "
Jul 28 20:53:22.033334 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:22.033126    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e21e71f1-41bb-4e12-965b-ae0d9bce305e-model-cache\") pod \"e21e71f1-41bb-4e12-965b-ae0d9bce305e\" (UID: \"e21e71f1-41bb-4e12-965b-ae0d9bce305e\") "
Jul 28 20:53:22.033334 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:22.033195    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e21e71f1-41bb-4e12-965b-ae0d9bce305e-tmp-dir\") pod \"e21e71f1-41bb-4e12-965b-ae0d9bce305e\" (UID: \"e21e71f1-41bb-4e12-965b-ae0d9bce305e\") "
Jul 28 20:53:22.033334 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:22.033244    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e21e71f1-41bb-4e12-965b-ae0d9bce305e-home" (OuterVolumeSpecName: "home") pod "e21e71f1-41bb-4e12-965b-ae0d9bce305e" (UID: "e21e71f1-41bb-4e12-965b-ae0d9bce305e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:53:22.033667 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:22.033365    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e21e71f1-41bb-4e12-965b-ae0d9bce305e-model-cache" (OuterVolumeSpecName: "model-cache") pod "e21e71f1-41bb-4e12-965b-ae0d9bce305e" (UID: "e21e71f1-41bb-4e12-965b-ae0d9bce305e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:53:22.033667 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:22.033503    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e21e71f1-41bb-4e12-965b-ae0d9bce305e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e21e71f1-41bb-4e12-965b-ae0d9bce305e" (UID: "e21e71f1-41bb-4e12-965b-ae0d9bce305e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:53:22.033667 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:22.033551    2554 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e21e71f1-41bb-4e12-965b-ae0d9bce305e-home\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:53:22.033667 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:22.033565    2554 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e21e71f1-41bb-4e12-965b-ae0d9bce305e-model-cache\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:53:22.035060 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:22.035036    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e21e71f1-41bb-4e12-965b-ae0d9bce305e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e21e71f1-41bb-4e12-965b-ae0d9bce305e" (UID: "e21e71f1-41bb-4e12-965b-ae0d9bce305e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:53:22.035472 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:22.035447    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e21e71f1-41bb-4e12-965b-ae0d9bce305e-dshm" (OuterVolumeSpecName: "dshm") pod "e21e71f1-41bb-4e12-965b-ae0d9bce305e" (UID: "e21e71f1-41bb-4e12-965b-ae0d9bce305e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:53:22.035574 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:22.035476    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e21e71f1-41bb-4e12-965b-ae0d9bce305e-kube-api-access-czs5j" (OuterVolumeSpecName: "kube-api-access-czs5j") pod "e21e71f1-41bb-4e12-965b-ae0d9bce305e" (UID: "e21e71f1-41bb-4e12-965b-ae0d9bce305e"). InnerVolumeSpecName "kube-api-access-czs5j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:53:22.134405 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:22.134321    2554 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e21e71f1-41bb-4e12-965b-ae0d9bce305e-dshm\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:53:22.134405 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:22.134349    2554 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e21e71f1-41bb-4e12-965b-ae0d9bce305e-tls-certs\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:53:22.134405 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:22.134359    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-czs5j\" (UniqueName: \"kubernetes.io/projected/e21e71f1-41bb-4e12-965b-ae0d9bce305e-kube-api-access-czs5j\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:53:22.134405 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:22.134367    2554 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e21e71f1-41bb-4e12-965b-ae0d9bce305e-tmp-dir\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:53:22.554268 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:22.554236    2554 generic.go:356] "Generic (PLEG): container finished" podID="e21e71f1-41bb-4e12-965b-ae0d9bce305e" containerID="f4121eb1bbaf95e240c53565b75d4a31c5dfa809fc1f47c5c82aefab1c775c39" exitCode=0
Jul 28 20:53:22.554690 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:22.554277    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c0e826/rate-test-model-c0e826-kserve-f9cf84d75-j7xls"
Jul 28 20:53:22.554690 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:22.554317    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c0e826/rate-test-model-c0e826-kserve-f9cf84d75-j7xls" event={"ID":"e21e71f1-41bb-4e12-965b-ae0d9bce305e","Type":"ContainerDied","Data":"f4121eb1bbaf95e240c53565b75d4a31c5dfa809fc1f47c5c82aefab1c775c39"}
Jul 28 20:53:22.554690 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:22.554360    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c0e826/rate-test-model-c0e826-kserve-f9cf84d75-j7xls" event={"ID":"e21e71f1-41bb-4e12-965b-ae0d9bce305e","Type":"ContainerDied","Data":"44cc08205a6c191235b6a29e00950ba5895ddf72ff272ca2ba0221658eb1893e"}
Jul 28 20:53:22.554690 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:22.554382    2554 scope.go:117] "RemoveContainer" containerID="f4121eb1bbaf95e240c53565b75d4a31c5dfa809fc1f47c5c82aefab1c775c39"
Jul 28 20:53:22.563316 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:22.563295    2554 scope.go:117] "RemoveContainer" containerID="f4121eb1bbaf95e240c53565b75d4a31c5dfa809fc1f47c5c82aefab1c775c39"
Jul 28 20:53:22.563607 ip-10-0-139-54 kubenswrapper[2554]: E0728 20:53:22.563588    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4121eb1bbaf95e240c53565b75d4a31c5dfa809fc1f47c5c82aefab1c775c39\": container with ID starting with f4121eb1bbaf95e240c53565b75d4a31c5dfa809fc1f47c5c82aefab1c775c39 not found: ID does not exist" containerID="f4121eb1bbaf95e240c53565b75d4a31c5dfa809fc1f47c5c82aefab1c775c39"
Jul 28 20:53:22.563676 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:22.563617    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4121eb1bbaf95e240c53565b75d4a31c5dfa809fc1f47c5c82aefab1c775c39"} err="failed to get container status \"f4121eb1bbaf95e240c53565b75d4a31c5dfa809fc1f47c5c82aefab1c775c39\": rpc error: code = NotFound desc = could not find container \"f4121eb1bbaf95e240c53565b75d4a31c5dfa809fc1f47c5c82aefab1c775c39\": container with ID starting with f4121eb1bbaf95e240c53565b75d4a31c5dfa809fc1f47c5c82aefab1c775c39 not found: ID does not exist"
Jul 28 20:53:22.589805 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:22.589765    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c0e826/rate-test-model-c0e826-kserve-f9cf84d75-j7xls"]
Jul 28 20:53:22.602475 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:22.602440    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-c0e826/rate-test-model-c0e826-kserve-f9cf84d75-j7xls"]
Jul 28 20:53:22.687214 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:22.687192    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7507-139960181992448"
Jul 28 20:53:22.728285 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:22.728256    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7507-139960181992448_1f072f26-1a86-4134-8e65-7c53d77d74e4/test-curl-7507-139960181992448/0.log"
Jul 28 20:53:22.758245 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:22.758213    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7507-139960181992448"]
Jul 28 20:53:22.762481 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:22.762458    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7507-139960181992448"]
Jul 28 20:53:22.841063 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:22.840981    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gp66n\" (UniqueName: \"kubernetes.io/projected/1f072f26-1a86-4134-8e65-7c53d77d74e4-kube-api-access-gp66n\") pod \"1f072f26-1a86-4134-8e65-7c53d77d74e4\" (UID: \"1f072f26-1a86-4134-8e65-7c53d77d74e4\") "
Jul 28 20:53:22.843054 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:22.843025    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f072f26-1a86-4134-8e65-7c53d77d74e4-kube-api-access-gp66n" (OuterVolumeSpecName: "kube-api-access-gp66n") pod "1f072f26-1a86-4134-8e65-7c53d77d74e4" (UID: "1f072f26-1a86-4134-8e65-7c53d77d74e4"). InnerVolumeSpecName "kube-api-access-gp66n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:53:22.941626 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:22.941593    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gp66n\" (UniqueName: \"kubernetes.io/projected/1f072f26-1a86-4134-8e65-7c53d77d74e4-kube-api-access-gp66n\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:53:23.561646 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:23.561616    2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e11cc934958611be4bee7cc73f7f77b9071d705c632791bda7faa7066120c3bd"
Jul 28 20:53:23.561646 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:23.561632    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7507-139960181992448"
Jul 28 20:53:23.572218 ip-10-0-139-54 kubenswrapper[2554]: E0728 20:53:23.572189    2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7507-139960181992448\" is forbidden: User \"system:node:ip-10-0-139-54.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-54.ec2.internal' and this object" podUID="1f072f26-1a86-4134-8e65-7c53d77d74e4" pod="opendatahub/test-curl-7507-139960181992448"
Jul 28 20:53:23.777345 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:23.777311    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f072f26-1a86-4134-8e65-7c53d77d74e4" path="/var/lib/kubelet/pods/1f072f26-1a86-4134-8e65-7c53d77d74e4/volumes"
Jul 28 20:53:23.777701 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:23.777689    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e21e71f1-41bb-4e12-965b-ae0d9bce305e" path="/var/lib/kubelet/pods/e21e71f1-41bb-4e12-965b-ae0d9bce305e/volumes"
Jul 28 20:53:50.238295 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:50.238259    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-1ac855-66d77475b8-66wbc"]
Jul 28 20:53:50.238762 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:50.238599    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f072f26-1a86-4134-8e65-7c53d77d74e4" containerName="test-curl-7507-139960181992448"
Jul 28 20:53:50.238762 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:50.238610    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f072f26-1a86-4134-8e65-7c53d77d74e4" containerName="test-curl-7507-139960181992448"
Jul 28 20:53:50.238762 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:50.238623    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e21e71f1-41bb-4e12-965b-ae0d9bce305e" containerName="main"
Jul 28 20:53:50.238762 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:50.238629    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="e21e71f1-41bb-4e12-965b-ae0d9bce305e" containerName="main"
Jul 28 20:53:50.238762 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:50.238682    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="e21e71f1-41bb-4e12-965b-ae0d9bce305e" containerName="main"
Jul 28 20:53:50.238762 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:50.238692    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f072f26-1a86-4134-8e65-7c53d77d74e4" containerName="test-curl-7507-139960181992448"
Jul 28 20:53:50.241816 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:50.241799    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1ac855-66d77475b8-66wbc"
Jul 28 20:53:50.244644 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:50.244618    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-1ac855-serving-cert\""
Jul 28 20:53:50.244644 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:50.244641    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-bqrm9\""
Jul 28 20:53:50.244889 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:50.244876    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 20:53:50.245995 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:50.245982    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 28 20:53:50.251489 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:50.251464    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-1ac855-66d77475b8-66wbc"]
Jul 28 20:53:50.290741 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:50.290706    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-1ac855-66d77475b8-66wbc\" (UID: \"d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1ac855-66d77475b8-66wbc"
Jul 28 20:53:50.290920 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:50.290843    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9hzv\" (UniqueName: \"kubernetes.io/projected/d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff-kube-api-access-v9hzv\") pod \"maas-api-e2e-ipp-b-1ac855-66d77475b8-66wbc\" (UID: \"d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1ac855-66d77475b8-66wbc"
Jul 28 20:53:50.290969 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:50.290922    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff-maas-api-tls\") pod \"maas-api-e2e-ipp-b-1ac855-66d77475b8-66wbc\" (UID: \"d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1ac855-66d77475b8-66wbc"
Jul 28 20:53:50.391533 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:50.391493    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9hzv\" (UniqueName: \"kubernetes.io/projected/d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff-kube-api-access-v9hzv\") pod \"maas-api-e2e-ipp-b-1ac855-66d77475b8-66wbc\" (UID: \"d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1ac855-66d77475b8-66wbc"
Jul 28 20:53:50.391709 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:50.391547    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff-maas-api-tls\") pod \"maas-api-e2e-ipp-b-1ac855-66d77475b8-66wbc\" (UID: \"d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1ac855-66d77475b8-66wbc"
Jul 28 20:53:50.391709 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:50.391589    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-1ac855-66d77475b8-66wbc\" (UID: \"d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1ac855-66d77475b8-66wbc"
Jul 28 20:53:50.391832 ip-10-0-139-54 kubenswrapper[2554]: E0728 20:53:50.391708    2554 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-1ac855-serving-cert: secret "maas-api-e2e-ipp-b-1ac855-serving-cert" not found
Jul 28 20:53:50.391832 ip-10-0-139-54 kubenswrapper[2554]: E0728 20:53:50.391781    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff-maas-api-tls podName:d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff nodeName:}" failed. No retries permitted until 2026-07-28 20:53:50.891765406 +0000 UTC m=+3293.674776826 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff-maas-api-tls") pod "maas-api-e2e-ipp-b-1ac855-66d77475b8-66wbc" (UID: "d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff") : secret "maas-api-e2e-ipp-b-1ac855-serving-cert" not found
Jul 28 20:53:50.392332 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:50.392301    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-1ac855-66d77475b8-66wbc\" (UID: \"d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1ac855-66d77475b8-66wbc"
Jul 28 20:53:50.400627 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:50.400609    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9hzv\" (UniqueName: \"kubernetes.io/projected/d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff-kube-api-access-v9hzv\") pod \"maas-api-e2e-ipp-b-1ac855-66d77475b8-66wbc\" (UID: \"d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1ac855-66d77475b8-66wbc"
Jul 28 20:53:50.895535 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:50.895489    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff-maas-api-tls\") pod \"maas-api-e2e-ipp-b-1ac855-66d77475b8-66wbc\" (UID: \"d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1ac855-66d77475b8-66wbc"
Jul 28 20:53:50.897850 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:50.897832    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff-maas-api-tls\") pod \"maas-api-e2e-ipp-b-1ac855-66d77475b8-66wbc\" (UID: \"d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1ac855-66d77475b8-66wbc"
Jul 28 20:53:51.153138 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:51.153023    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1ac855-66d77475b8-66wbc"
Jul 28 20:53:51.289027 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:51.288859    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-1ac855-66d77475b8-66wbc"]
Jul 28 20:53:51.291650 ip-10-0-139-54 kubenswrapper[2554]: W0728 20:53:51.291621    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7799548_ae2a_4e1c_bcf4_5747b8d1d1ff.slice/crio-a1d143d696e4b4101cfc6fe23b589e32ed8c822910071d58a9f5aca645b6556c WatchSource:0}: Error finding container a1d143d696e4b4101cfc6fe23b589e32ed8c822910071d58a9f5aca645b6556c: Status 404 returned error can't find the container with id a1d143d696e4b4101cfc6fe23b589e32ed8c822910071d58a9f5aca645b6556c
Jul 28 20:53:51.689452 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:51.689418    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1ac855-66d77475b8-66wbc" event={"ID":"d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff","Type":"ContainerStarted","Data":"a1d143d696e4b4101cfc6fe23b589e32ed8c822910071d58a9f5aca645b6556c"}
Jul 28 20:53:53.701209 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:53.701176    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1ac855-66d77475b8-66wbc" event={"ID":"d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff","Type":"ContainerStarted","Data":"3c446fdc03f71618b42ff863949c8ddcc7c5578d4ba72189931393075e72af99"}
Jul 28 20:53:53.701652 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:53.701310    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1ac855-66d77475b8-66wbc"
Jul 28 20:53:53.722192 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:53.722084    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1ac855-66d77475b8-66wbc" podStartSLOduration=1.557537803 podStartE2EDuration="3.722051801s" totalImagesPullingTime="2.164513998s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:53:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:53:53.721390471 +0000 UTC m=+3296.504401910" watchObservedRunningTime="2026-07-28 20:53:53.722051801 +0000 UTC m=+3296.505063247"
Jul 28 20:53:57.978258 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:57.978142    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-7vcr8_dde4c1a8-8229-4419-87b6-d8e39a0827c4/console-operator/1.log"
Jul 28 20:53:57.992287 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:57.981799    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-7vcr8_dde4c1a8-8229-4419-87b6-d8e39a0827c4/console-operator/1.log"
Jul 28 20:53:59.712044 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:53:59.712016    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1ac855-66d77475b8-66wbc"
Jul 28 20:54:27.681798 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:54:27.681757    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-2dfc5b/ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5"]
Jul 28 20:54:27.689530 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:54:27.689509    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-2dfc5b/ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5"
Jul 28 20:54:27.692474 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:54:27.692452    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-2dfc5b\"/\"openshift-service-ca.crt\""
Jul 28 20:54:27.694035 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:54:27.694016    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-2dfc5b\"/\"default-dockercfg-jr94x\""
Jul 28 20:54:27.694219 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:54:27.694194    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-2dfc5b\"/\"kube-root-ca.crt\""
Jul 28 20:54:27.694459 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:54:27.694437    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-2dfc5b\"/\"ipp-route-2dfc5b-kserve-self-signed-certs\""
Jul 28 20:54:27.696928 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:54:27.696419    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-2dfc5b/ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5"]
Jul 28 20:54:27.723406 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:54:27.723381    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kttt\" (UniqueName: \"kubernetes.io/projected/f3f441e9-955f-4cdd-907a-c771e9968e90-kube-api-access-9kttt\") pod \"ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5\" (UID: \"f3f441e9-955f-4cdd-907a-c771e9968e90\") " pod="ai-tenant-e2e-ipp-a-2dfc5b/ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5"
Jul 28 20:54:27.723536 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:54:27.723419    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3f441e9-955f-4cdd-907a-c771e9968e90-tmp-dir\") pod \"ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5\" (UID: \"f3f441e9-955f-4cdd-907a-c771e9968e90\") " pod="ai-tenant-e2e-ipp-a-2dfc5b/ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5"
Jul 28 20:54:27.723536 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:54:27.723455    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3f441e9-955f-4cdd-907a-c771e9968e90-home\") pod \"ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5\" (UID: \"f3f441e9-955f-4cdd-907a-c771e9968e90\") " pod="ai-tenant-e2e-ipp-a-2dfc5b/ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5"
Jul 28 20:54:27.723536 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:54:27.723501    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3f441e9-955f-4cdd-907a-c771e9968e90-model-cache\") pod \"ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5\" (UID: \"f3f441e9-955f-4cdd-907a-c771e9968e90\") " pod="ai-tenant-e2e-ipp-a-2dfc5b/ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5"
Jul 28 20:54:27.723662 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:54:27.723550    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3f441e9-955f-4cdd-907a-c771e9968e90-tls-certs\") pod \"ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5\" (UID: \"f3f441e9-955f-4cdd-907a-c771e9968e90\") " pod="ai-tenant-e2e-ipp-a-2dfc5b/ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5"
Jul 28 20:54:27.723662 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:54:27.723575    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3f441e9-955f-4cdd-907a-c771e9968e90-dshm\") pod \"ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5\" (UID: \"f3f441e9-955f-4cdd-907a-c771e9968e90\") " pod="ai-tenant-e2e-ipp-a-2dfc5b/ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5"
Jul 28 20:54:27.824051 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:54:27.824021    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3f441e9-955f-4cdd-907a-c771e9968e90-tls-certs\") pod \"ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5\" (UID: \"f3f441e9-955f-4cdd-907a-c771e9968e90\") " pod="ai-tenant-e2e-ipp-a-2dfc5b/ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5"
Jul 28 20:54:27.824225 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:54:27.824059    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3f441e9-955f-4cdd-907a-c771e9968e90-dshm\") pod \"ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5\" (UID: \"f3f441e9-955f-4cdd-907a-c771e9968e90\") " pod="ai-tenant-e2e-ipp-a-2dfc5b/ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5"
Jul 28 20:54:27.824225 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:54:27.824130    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kttt\" (UniqueName: \"kubernetes.io/projected/f3f441e9-955f-4cdd-907a-c771e9968e90-kube-api-access-9kttt\") pod \"ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5\" (UID: \"f3f441e9-955f-4cdd-907a-c771e9968e90\") " pod="ai-tenant-e2e-ipp-a-2dfc5b/ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5"
Jul 28 20:54:27.824225 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:54:27.824163    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3f441e9-955f-4cdd-907a-c771e9968e90-tmp-dir\") pod \"ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5\" (UID: \"f3f441e9-955f-4cdd-907a-c771e9968e90\") " pod="ai-tenant-e2e-ipp-a-2dfc5b/ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5"
Jul 28 20:54:27.824225 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:54:27.824202    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3f441e9-955f-4cdd-907a-c771e9968e90-home\") pod \"ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5\" (UID: \"f3f441e9-955f-4cdd-907a-c771e9968e90\") " pod="ai-tenant-e2e-ipp-a-2dfc5b/ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5"
Jul 28 20:54:27.824410 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:54:27.824233    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3f441e9-955f-4cdd-907a-c771e9968e90-model-cache\") pod \"ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5\" (UID: \"f3f441e9-955f-4cdd-907a-c771e9968e90\") " pod="ai-tenant-e2e-ipp-a-2dfc5b/ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5"
Jul 28 20:54:27.824653 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:54:27.824619    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3f441e9-955f-4cdd-907a-c771e9968e90-tmp-dir\") pod \"ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5\" (UID: \"f3f441e9-955f-4cdd-907a-c771e9968e90\") " pod="ai-tenant-e2e-ipp-a-2dfc5b/ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5"
Jul 28 20:54:27.824653 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:54:27.824632    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3f441e9-955f-4cdd-907a-c771e9968e90-home\") pod \"ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5\" (UID: \"f3f441e9-955f-4cdd-907a-c771e9968e90\") " pod="ai-tenant-e2e-ipp-a-2dfc5b/ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5"
Jul 28 20:54:27.824809 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:54:27.824663    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3f441e9-955f-4cdd-907a-c771e9968e90-model-cache\") pod \"ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5\" (UID: \"f3f441e9-955f-4cdd-907a-c771e9968e90\") " pod="ai-tenant-e2e-ipp-a-2dfc5b/ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5"
Jul 28 20:54:27.826340 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:54:27.826314    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3f441e9-955f-4cdd-907a-c771e9968e90-dshm\") pod \"ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5\" (UID: \"f3f441e9-955f-4cdd-907a-c771e9968e90\") " pod="ai-tenant-e2e-ipp-a-2dfc5b/ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5"
Jul 28 20:54:27.826644 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:54:27.826625    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3f441e9-955f-4cdd-907a-c771e9968e90-tls-certs\") pod \"ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5\" (UID: \"f3f441e9-955f-4cdd-907a-c771e9968e90\") " pod="ai-tenant-e2e-ipp-a-2dfc5b/ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5"
Jul 28 20:54:27.832931 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:54:27.832911    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kttt\" (UniqueName: \"kubernetes.io/projected/f3f441e9-955f-4cdd-907a-c771e9968e90-kube-api-access-9kttt\") pod \"ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5\" (UID: \"f3f441e9-955f-4cdd-907a-c771e9968e90\") " pod="ai-tenant-e2e-ipp-a-2dfc5b/ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5"
Jul 28 20:54:27.999445 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:54:27.999364    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-2dfc5b/ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5"
Jul 28 20:54:28.137916 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:54:28.137871    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-2dfc5b/ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5"]
Jul 28 20:54:28.140411 ip-10-0-139-54 kubenswrapper[2554]: W0728 20:54:28.140375    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf3f441e9_955f_4cdd_907a_c771e9968e90.slice/crio-a89aeda2f4221c8210f31c7409c26ba235000549d4f99b37d8b26dc7e695272f WatchSource:0}: Error finding container a89aeda2f4221c8210f31c7409c26ba235000549d4f99b37d8b26dc7e695272f: Status 404 returned error can't find the container with id a89aeda2f4221c8210f31c7409c26ba235000549d4f99b37d8b26dc7e695272f
Jul 28 20:54:28.871841 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:54:28.871802    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-2dfc5b/ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5" event={"ID":"f3f441e9-955f-4cdd-907a-c771e9968e90","Type":"ContainerStarted","Data":"d4a3501bf7d4ec918776d598cfb22666b233a98dd7449e4aff48a4c48d5f6161"}
Jul 28 20:54:28.871841 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:54:28.871845    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-2dfc5b/ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5" event={"ID":"f3f441e9-955f-4cdd-907a-c771e9968e90","Type":"ContainerStarted","Data":"a89aeda2f4221c8210f31c7409c26ba235000549d4f99b37d8b26dc7e695272f"}
Jul 28 20:54:28.889665 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:54:28.889618    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-2dfc5b/ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5" podStartSLOduration=1.889600779 podStartE2EDuration="1.889600779s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:54:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:54:28.888882397 +0000 UTC m=+3331.671893839" watchObservedRunningTime="2026-07-28 20:54:28.889600779 +0000 UTC m=+3331.672612223"
Jul 28 20:54:37.999459 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:54:37.999419    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-2dfc5b/ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5"
Jul 28 20:54:37.999926 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:54:37.999471    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-2dfc5b/ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5"
Jul 28 20:54:38.011967 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:54:38.011933    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-2dfc5b/ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5"
Jul 28 20:54:38.932405 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:54:38.932372    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-2dfc5b/ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5"
Jul 28 20:55:18.701523 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:18.701485    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d6a3fd-76c7bccb9-gj8ls"]
Jul 28 20:55:18.705089 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:18.705054    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d6a3fd-76c7bccb9-gj8ls"
Jul 28 20:55:18.709847 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:18.709811    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-d6a3fd-serving-cert\""
Jul 28 20:55:18.721774 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:18.721746    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d6a3fd-76c7bccb9-gj8ls"]
Jul 28 20:55:18.799724 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:18.799673    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/682a174e-e22f-44b0-9dfe-49ab56b00476-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-d6a3fd-76c7bccb9-gj8ls\" (UID: \"682a174e-e22f-44b0-9dfe-49ab56b00476\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d6a3fd-76c7bccb9-gj8ls"
Jul 28 20:55:18.799724 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:18.799725    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mssnz\" (UniqueName: \"kubernetes.io/projected/682a174e-e22f-44b0-9dfe-49ab56b00476-kube-api-access-mssnz\") pod \"maas-api-e2e-ipp-cleanup-d6a3fd-76c7bccb9-gj8ls\" (UID: \"682a174e-e22f-44b0-9dfe-49ab56b00476\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d6a3fd-76c7bccb9-gj8ls"
Jul 28 20:55:18.799993 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:18.799881    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/682a174e-e22f-44b0-9dfe-49ab56b00476-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-d6a3fd-76c7bccb9-gj8ls\" (UID: \"682a174e-e22f-44b0-9dfe-49ab56b00476\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d6a3fd-76c7bccb9-gj8ls"
Jul 28 20:55:18.901353 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:18.901303    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/682a174e-e22f-44b0-9dfe-49ab56b00476-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-d6a3fd-76c7bccb9-gj8ls\" (UID: \"682a174e-e22f-44b0-9dfe-49ab56b00476\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d6a3fd-76c7bccb9-gj8ls"
Jul 28 20:55:18.901556 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:18.901396    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/682a174e-e22f-44b0-9dfe-49ab56b00476-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-d6a3fd-76c7bccb9-gj8ls\" (UID: \"682a174e-e22f-44b0-9dfe-49ab56b00476\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d6a3fd-76c7bccb9-gj8ls"
Jul 28 20:55:18.901556 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:18.901416    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mssnz\" (UniqueName: \"kubernetes.io/projected/682a174e-e22f-44b0-9dfe-49ab56b00476-kube-api-access-mssnz\") pod \"maas-api-e2e-ipp-cleanup-d6a3fd-76c7bccb9-gj8ls\" (UID: \"682a174e-e22f-44b0-9dfe-49ab56b00476\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d6a3fd-76c7bccb9-gj8ls"
Jul 28 20:55:18.901691 ip-10-0-139-54 kubenswrapper[2554]: E0728 20:55:18.901570    2554 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d6a3fd-serving-cert: secret "maas-api-e2e-ipp-cleanup-d6a3fd-serving-cert" not found
Jul 28 20:55:18.901691 ip-10-0-139-54 kubenswrapper[2554]: E0728 20:55:18.901679    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/682a174e-e22f-44b0-9dfe-49ab56b00476-maas-api-tls podName:682a174e-e22f-44b0-9dfe-49ab56b00476 nodeName:}" failed. No retries permitted until 2026-07-28 20:55:19.401655134 +0000 UTC m=+3382.184666564 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/682a174e-e22f-44b0-9dfe-49ab56b00476-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-d6a3fd-76c7bccb9-gj8ls" (UID: "682a174e-e22f-44b0-9dfe-49ab56b00476") : secret "maas-api-e2e-ipp-cleanup-d6a3fd-serving-cert" not found
Jul 28 20:55:18.902045 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:18.902019    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/682a174e-e22f-44b0-9dfe-49ab56b00476-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-d6a3fd-76c7bccb9-gj8ls\" (UID: \"682a174e-e22f-44b0-9dfe-49ab56b00476\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d6a3fd-76c7bccb9-gj8ls"
Jul 28 20:55:18.910183 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:18.910162    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mssnz\" (UniqueName: \"kubernetes.io/projected/682a174e-e22f-44b0-9dfe-49ab56b00476-kube-api-access-mssnz\") pod \"maas-api-e2e-ipp-cleanup-d6a3fd-76c7bccb9-gj8ls\" (UID: \"682a174e-e22f-44b0-9dfe-49ab56b00476\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d6a3fd-76c7bccb9-gj8ls"
Jul 28 20:55:19.407163 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:19.407127    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/682a174e-e22f-44b0-9dfe-49ab56b00476-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-d6a3fd-76c7bccb9-gj8ls\" (UID: \"682a174e-e22f-44b0-9dfe-49ab56b00476\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d6a3fd-76c7bccb9-gj8ls"
Jul 28 20:55:19.409705 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:19.409680    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/682a174e-e22f-44b0-9dfe-49ab56b00476-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-d6a3fd-76c7bccb9-gj8ls\" (UID: \"682a174e-e22f-44b0-9dfe-49ab56b00476\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d6a3fd-76c7bccb9-gj8ls"
Jul 28 20:55:19.615435 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:19.615391    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d6a3fd-76c7bccb9-gj8ls"
Jul 28 20:55:19.960104 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:19.960064    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d6a3fd-76c7bccb9-gj8ls"]
Jul 28 20:55:19.961996 ip-10-0-139-54 kubenswrapper[2554]: W0728 20:55:19.961967    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod682a174e_e22f_44b0_9dfe_49ab56b00476.slice/crio-6d4b620636f984ed9004c573e96bca25fb27c7115f4404c0d0099c87b89da0fd WatchSource:0}: Error finding container 6d4b620636f984ed9004c573e96bca25fb27c7115f4404c0d0099c87b89da0fd: Status 404 returned error can't find the container with id 6d4b620636f984ed9004c573e96bca25fb27c7115f4404c0d0099c87b89da0fd
Jul 28 20:55:20.125207 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:20.125172    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d6a3fd-76c7bccb9-gj8ls" event={"ID":"682a174e-e22f-44b0-9dfe-49ab56b00476","Type":"ContainerStarted","Data":"6d4b620636f984ed9004c573e96bca25fb27c7115f4404c0d0099c87b89da0fd"}
Jul 28 20:55:21.130986 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:21.130947    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d6a3fd-76c7bccb9-gj8ls" event={"ID":"682a174e-e22f-44b0-9dfe-49ab56b00476","Type":"ContainerStarted","Data":"d9e39efa8a5e0f7ad0d6a5b08ef16d7c368404efd1f6760b92724424e4033864"}
Jul 28 20:55:21.131440 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:21.131116    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d6a3fd-76c7bccb9-gj8ls"
Jul 28 20:55:21.148335 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:21.148280    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d6a3fd-76c7bccb9-gj8ls" podStartSLOduration=2.572263615 podStartE2EDuration="3.148267336s" totalImagesPullingTime="576.003721ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:55:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:55:21.147875 +0000 UTC m=+3383.930886442" watchObservedRunningTime="2026-07-28 20:55:21.148267336 +0000 UTC m=+3383.931278778"
Jul 28 20:55:27.141087 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:27.141039    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d6a3fd-76c7bccb9-gj8ls"
Jul 28 20:55:37.169628 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:37.169595    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d6a3fd-xvt9j"]
Jul 28 20:55:37.173239 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:37.173222    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d6a3fd-xvt9j"
Jul 28 20:55:37.175978 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:37.175951    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-982zh\""
Jul 28 20:55:37.180512 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:37.180485    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d6a3fd-xvt9j"]
Jul 28 20:55:37.276296 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:37.276260    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/949574f9-d672-4509-9263-646f7df3613c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-d6a3fd-xvt9j\" (UID: \"949574f9-d672-4509-9263-646f7df3613c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d6a3fd-xvt9j"
Jul 28 20:55:37.377133 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:37.377093    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/949574f9-d672-4509-9263-646f7df3613c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-d6a3fd-xvt9j\" (UID: \"949574f9-d672-4509-9263-646f7df3613c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d6a3fd-xvt9j"
Jul 28 20:55:37.377717 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:37.377698    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/949574f9-d672-4509-9263-646f7df3613c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-d6a3fd-xvt9j\" (UID: \"949574f9-d672-4509-9263-646f7df3613c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d6a3fd-xvt9j"
Jul 28 20:55:37.485024 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:37.484928    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d6a3fd-xvt9j"
Jul 28 20:55:37.614986 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:37.614957    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d6a3fd-xvt9j"]
Jul 28 20:55:37.616778 ip-10-0-139-54 kubenswrapper[2554]: W0728 20:55:37.616752    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod949574f9_d672_4509_9263_646f7df3613c.slice/crio-7accf48f2bd345045832f2f2969264dbcf5bc264da68f157f021e31b4ad4bafa WatchSource:0}: Error finding container 7accf48f2bd345045832f2f2969264dbcf5bc264da68f157f021e31b4ad4bafa: Status 404 returned error can't find the container with id 7accf48f2bd345045832f2f2969264dbcf5bc264da68f157f021e31b4ad4bafa
Jul 28 20:55:38.214051 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:38.214014    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d6a3fd-xvt9j" event={"ID":"949574f9-d672-4509-9263-646f7df3613c","Type":"ContainerStarted","Data":"7accf48f2bd345045832f2f2969264dbcf5bc264da68f157f021e31b4ad4bafa"}
Jul 28 20:55:41.242358 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:41.242270    2554 generic.go:356] "Generic (PLEG): container finished" podID="949574f9-d672-4509-9263-646f7df3613c" containerID="71bc09cb263604a0d03b4e76fd4a3491e3bf90adc9ae7497108f6ea6d315967d" exitCode=0
Jul 28 20:55:41.242358 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:41.242350    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d6a3fd-xvt9j" event={"ID":"949574f9-d672-4509-9263-646f7df3613c","Type":"ContainerDied","Data":"71bc09cb263604a0d03b4e76fd4a3491e3bf90adc9ae7497108f6ea6d315967d"}
Jul 28 20:55:42.387677 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:42.387651    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d6a3fd-xvt9j"
Jul 28 20:55:42.416113 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:42.416065    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/949574f9-d672-4509-9263-646f7df3613c-maas-api-service-ca\") pod \"949574f9-d672-4509-9263-646f7df3613c\" (UID: \"949574f9-d672-4509-9263-646f7df3613c\") "
Jul 28 20:55:42.416418 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:42.416398    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/949574f9-d672-4509-9263-646f7df3613c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "949574f9-d672-4509-9263-646f7df3613c" (UID: "949574f9-d672-4509-9263-646f7df3613c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:55:42.517120 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:42.517015    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/949574f9-d672-4509-9263-646f7df3613c-maas-api-service-ca\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:55:43.253284 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:43.253250    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d6a3fd-xvt9j"
Jul 28 20:55:43.253452 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:43.253285    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d6a3fd-xvt9j" event={"ID":"949574f9-d672-4509-9263-646f7df3613c","Type":"ContainerDied","Data":"7accf48f2bd345045832f2f2969264dbcf5bc264da68f157f021e31b4ad4bafa"}
Jul 28 20:55:43.253452 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:43.253335    2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7accf48f2bd345045832f2f2969264dbcf5bc264da68f157f021e31b4ad4bafa"
Jul 28 20:55:49.136235 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:49.136191    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d6a3fd-76c7bccb9-gj8ls"]
Jul 28 20:55:49.136735 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:49.136439    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d6a3fd-76c7bccb9-gj8ls" podUID="682a174e-e22f-44b0-9dfe-49ab56b00476" containerName="maas-api" containerID="cri-o://d9e39efa8a5e0f7ad0d6a5b08ef16d7c368404efd1f6760b92724424e4033864" gracePeriod=30
Jul 28 20:55:49.283378 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:49.283340    2554 generic.go:356] "Generic (PLEG): container finished" podID="682a174e-e22f-44b0-9dfe-49ab56b00476" containerID="d9e39efa8a5e0f7ad0d6a5b08ef16d7c368404efd1f6760b92724424e4033864" exitCode=0
Jul 28 20:55:49.283557 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:49.283445    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d6a3fd-76c7bccb9-gj8ls" event={"ID":"682a174e-e22f-44b0-9dfe-49ab56b00476","Type":"ContainerDied","Data":"d9e39efa8a5e0f7ad0d6a5b08ef16d7c368404efd1f6760b92724424e4033864"}
Jul 28 20:55:49.389564 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:49.389505    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d6a3fd-76c7bccb9-gj8ls"
Jul 28 20:55:49.479118 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:49.479059    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mssnz\" (UniqueName: \"kubernetes.io/projected/682a174e-e22f-44b0-9dfe-49ab56b00476-kube-api-access-mssnz\") pod \"682a174e-e22f-44b0-9dfe-49ab56b00476\" (UID: \"682a174e-e22f-44b0-9dfe-49ab56b00476\") "
Jul 28 20:55:49.479319 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:49.479143    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/682a174e-e22f-44b0-9dfe-49ab56b00476-openshift-service-ca\") pod \"682a174e-e22f-44b0-9dfe-49ab56b00476\" (UID: \"682a174e-e22f-44b0-9dfe-49ab56b00476\") "
Jul 28 20:55:49.479319 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:49.479200    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/682a174e-e22f-44b0-9dfe-49ab56b00476-maas-api-tls\") pod \"682a174e-e22f-44b0-9dfe-49ab56b00476\" (UID: \"682a174e-e22f-44b0-9dfe-49ab56b00476\") "
Jul 28 20:55:49.479570 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:49.479546    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/682a174e-e22f-44b0-9dfe-49ab56b00476-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "682a174e-e22f-44b0-9dfe-49ab56b00476" (UID: "682a174e-e22f-44b0-9dfe-49ab56b00476"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:55:49.481470 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:49.481441    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/682a174e-e22f-44b0-9dfe-49ab56b00476-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "682a174e-e22f-44b0-9dfe-49ab56b00476" (UID: "682a174e-e22f-44b0-9dfe-49ab56b00476"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:55:49.481555 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:49.481447    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/682a174e-e22f-44b0-9dfe-49ab56b00476-kube-api-access-mssnz" (OuterVolumeSpecName: "kube-api-access-mssnz") pod "682a174e-e22f-44b0-9dfe-49ab56b00476" (UID: "682a174e-e22f-44b0-9dfe-49ab56b00476"). InnerVolumeSpecName "kube-api-access-mssnz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:55:49.580006 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:49.579966    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mssnz\" (UniqueName: \"kubernetes.io/projected/682a174e-e22f-44b0-9dfe-49ab56b00476-kube-api-access-mssnz\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:55:49.580006 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:49.579999    2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/682a174e-e22f-44b0-9dfe-49ab56b00476-openshift-service-ca\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:55:49.580006 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:49.580010    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/682a174e-e22f-44b0-9dfe-49ab56b00476-maas-api-tls\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:55:49.595058 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:49.595028    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d6a3fd-xvt9j"]
Jul 28 20:55:49.599113 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:49.599085    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-d6a3fd-xvt9j"]
Jul 28 20:55:49.777684 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:49.777575    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="949574f9-d672-4509-9263-646f7df3613c" path="/var/lib/kubelet/pods/949574f9-d672-4509-9263-646f7df3613c/volumes"
Jul 28 20:55:50.289302 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:50.289272    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d6a3fd-76c7bccb9-gj8ls"
Jul 28 20:55:50.289687 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:50.289271    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d6a3fd-76c7bccb9-gj8ls" event={"ID":"682a174e-e22f-44b0-9dfe-49ab56b00476","Type":"ContainerDied","Data":"6d4b620636f984ed9004c573e96bca25fb27c7115f4404c0d0099c87b89da0fd"}
Jul 28 20:55:50.289687 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:50.289403    2554 scope.go:117] "RemoveContainer" containerID="d9e39efa8a5e0f7ad0d6a5b08ef16d7c368404efd1f6760b92724424e4033864"
Jul 28 20:55:50.323087 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:50.323040    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d6a3fd-76c7bccb9-gj8ls"]
Jul 28 20:55:50.330769 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:50.330741    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d6a3fd-76c7bccb9-gj8ls"]
Jul 28 20:55:51.777697 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:55:51.777662    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="682a174e-e22f-44b0-9dfe-49ab56b00476" path="/var/lib/kubelet/pods/682a174e-e22f-44b0-9dfe-49ab56b00476/volumes"
Jul 28 20:56:24.939869 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:56:24.939840    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-2dfc5b/ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5"]
Jul 28 20:56:39.959275 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:56:39.959205    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-2dfc5b/ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5" podUID="f3f441e9-955f-4cdd-907a-c771e9968e90" containerName="main" containerID="cri-o://d4a3501bf7d4ec918776d598cfb22666b233a98dd7449e4aff48a4c48d5f6161" gracePeriod=45
Jul 28 20:56:40.218651 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:56:40.218589    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-2dfc5b/ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5"
Jul 28 20:56:40.340804 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:56:40.340765    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3f441e9-955f-4cdd-907a-c771e9968e90-tmp-dir\") pod \"f3f441e9-955f-4cdd-907a-c771e9968e90\" (UID: \"f3f441e9-955f-4cdd-907a-c771e9968e90\") "
Jul 28 20:56:40.341000 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:56:40.340867    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9kttt\" (UniqueName: \"kubernetes.io/projected/f3f441e9-955f-4cdd-907a-c771e9968e90-kube-api-access-9kttt\") pod \"f3f441e9-955f-4cdd-907a-c771e9968e90\" (UID: \"f3f441e9-955f-4cdd-907a-c771e9968e90\") "
Jul 28 20:56:40.341000 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:56:40.340895    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3f441e9-955f-4cdd-907a-c771e9968e90-tls-certs\") pod \"f3f441e9-955f-4cdd-907a-c771e9968e90\" (UID: \"f3f441e9-955f-4cdd-907a-c771e9968e90\") "
Jul 28 20:56:40.341000 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:56:40.340915    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3f441e9-955f-4cdd-907a-c771e9968e90-dshm\") pod \"f3f441e9-955f-4cdd-907a-c771e9968e90\" (UID: \"f3f441e9-955f-4cdd-907a-c771e9968e90\") "
Jul 28 20:56:40.341000 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:56:40.340945    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3f441e9-955f-4cdd-907a-c771e9968e90-home\") pod \"f3f441e9-955f-4cdd-907a-c771e9968e90\" (UID: \"f3f441e9-955f-4cdd-907a-c771e9968e90\") "
Jul 28 20:56:40.341239 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:56:40.341060    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3f441e9-955f-4cdd-907a-c771e9968e90-model-cache\") pod \"f3f441e9-955f-4cdd-907a-c771e9968e90\" (UID: \"f3f441e9-955f-4cdd-907a-c771e9968e90\") "
Jul 28 20:56:40.341239 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:56:40.341055    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3f441e9-955f-4cdd-907a-c771e9968e90-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f3f441e9-955f-4cdd-907a-c771e9968e90" (UID: "f3f441e9-955f-4cdd-907a-c771e9968e90"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:56:40.341359 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:56:40.341292    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3f441e9-955f-4cdd-907a-c771e9968e90-home" (OuterVolumeSpecName: "home") pod "f3f441e9-955f-4cdd-907a-c771e9968e90" (UID: "f3f441e9-955f-4cdd-907a-c771e9968e90"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:56:40.341359 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:56:40.341312    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3f441e9-955f-4cdd-907a-c771e9968e90-model-cache" (OuterVolumeSpecName: "model-cache") pod "f3f441e9-955f-4cdd-907a-c771e9968e90" (UID: "f3f441e9-955f-4cdd-907a-c771e9968e90"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:56:40.341463 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:56:40.341424    2554 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3f441e9-955f-4cdd-907a-c771e9968e90-model-cache\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:56:40.341463 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:56:40.341438    2554 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3f441e9-955f-4cdd-907a-c771e9968e90-tmp-dir\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:56:40.341463 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:56:40.341450    2554 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3f441e9-955f-4cdd-907a-c771e9968e90-home\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:56:40.343149 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:56:40.343124    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3f441e9-955f-4cdd-907a-c771e9968e90-dshm" (OuterVolumeSpecName: "dshm") pod "f3f441e9-955f-4cdd-907a-c771e9968e90" (UID: "f3f441e9-955f-4cdd-907a-c771e9968e90"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 20:56:40.343261 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:56:40.343165    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3f441e9-955f-4cdd-907a-c771e9968e90-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f3f441e9-955f-4cdd-907a-c771e9968e90" (UID: "f3f441e9-955f-4cdd-907a-c771e9968e90"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:56:40.343261 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:56:40.343204    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3f441e9-955f-4cdd-907a-c771e9968e90-kube-api-access-9kttt" (OuterVolumeSpecName: "kube-api-access-9kttt") pod "f3f441e9-955f-4cdd-907a-c771e9968e90" (UID: "f3f441e9-955f-4cdd-907a-c771e9968e90"). InnerVolumeSpecName "kube-api-access-9kttt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:56:40.441903 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:56:40.441865    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9kttt\" (UniqueName: \"kubernetes.io/projected/f3f441e9-955f-4cdd-907a-c771e9968e90-kube-api-access-9kttt\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:56:40.441903 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:56:40.441894    2554 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3f441e9-955f-4cdd-907a-c771e9968e90-tls-certs\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:56:40.441903 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:56:40.441905    2554 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3f441e9-955f-4cdd-907a-c771e9968e90-dshm\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:56:40.531026 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:56:40.530934    2554 generic.go:356] "Generic (PLEG): container finished" podID="f3f441e9-955f-4cdd-907a-c771e9968e90" containerID="d4a3501bf7d4ec918776d598cfb22666b233a98dd7449e4aff48a4c48d5f6161" exitCode=0
Jul 28 20:56:40.531026 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:56:40.530973    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-2dfc5b/ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5"
Jul 28 20:56:40.531026 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:56:40.531010    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-2dfc5b/ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5" event={"ID":"f3f441e9-955f-4cdd-907a-c771e9968e90","Type":"ContainerDied","Data":"d4a3501bf7d4ec918776d598cfb22666b233a98dd7449e4aff48a4c48d5f6161"}
Jul 28 20:56:40.531304 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:56:40.531049    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-2dfc5b/ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5" event={"ID":"f3f441e9-955f-4cdd-907a-c771e9968e90","Type":"ContainerDied","Data":"a89aeda2f4221c8210f31c7409c26ba235000549d4f99b37d8b26dc7e695272f"}
Jul 28 20:56:40.531304 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:56:40.531099    2554 scope.go:117] "RemoveContainer" containerID="d4a3501bf7d4ec918776d598cfb22666b233a98dd7449e4aff48a4c48d5f6161"
Jul 28 20:56:40.540089 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:56:40.540054    2554 scope.go:117] "RemoveContainer" containerID="d4a3501bf7d4ec918776d598cfb22666b233a98dd7449e4aff48a4c48d5f6161"
Jul 28 20:56:40.540331 ip-10-0-139-54 kubenswrapper[2554]: E0728 20:56:40.540311    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4a3501bf7d4ec918776d598cfb22666b233a98dd7449e4aff48a4c48d5f6161\": container with ID starting with d4a3501bf7d4ec918776d598cfb22666b233a98dd7449e4aff48a4c48d5f6161 not found: ID does not exist" containerID="d4a3501bf7d4ec918776d598cfb22666b233a98dd7449e4aff48a4c48d5f6161"
Jul 28 20:56:40.540386 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:56:40.540340    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4a3501bf7d4ec918776d598cfb22666b233a98dd7449e4aff48a4c48d5f6161"} err="failed to get container status \"d4a3501bf7d4ec918776d598cfb22666b233a98dd7449e4aff48a4c48d5f6161\": rpc error: code = NotFound desc = could not find container \"d4a3501bf7d4ec918776d598cfb22666b233a98dd7449e4aff48a4c48d5f6161\": container with ID starting with d4a3501bf7d4ec918776d598cfb22666b233a98dd7449e4aff48a4c48d5f6161 not found: ID does not exist"
Jul 28 20:56:40.554644 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:56:40.554621    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-2dfc5b/ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5"]
Jul 28 20:56:40.557879 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:56:40.557858    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-2dfc5b/ipp-route-2dfc5b-kserve-64f46d95d7-6mtt5"]
Jul 28 20:56:41.777964 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:56:41.777927    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3f441e9-955f-4cdd-907a-c771e9968e90" path="/var/lib/kubelet/pods/f3f441e9-955f-4cdd-907a-c771e9968e90/volumes"
Jul 28 20:57:09.408918 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:09.408883    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-1ac855-zm67h"]
Jul 28 20:57:09.409385 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:09.409237    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="949574f9-d672-4509-9263-646f7df3613c" containerName="revoke-keys"
Jul 28 20:57:09.409385 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:09.409248    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="949574f9-d672-4509-9263-646f7df3613c" containerName="revoke-keys"
Jul 28 20:57:09.409385 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:09.409259    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="682a174e-e22f-44b0-9dfe-49ab56b00476" containerName="maas-api"
Jul 28 20:57:09.409385 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:09.409264    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="682a174e-e22f-44b0-9dfe-49ab56b00476" containerName="maas-api"
Jul 28 20:57:09.409385 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:09.409282    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3f441e9-955f-4cdd-907a-c771e9968e90" containerName="main"
Jul 28 20:57:09.409385 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:09.409288    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3f441e9-955f-4cdd-907a-c771e9968e90" containerName="main"
Jul 28 20:57:09.409385 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:09.409350    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3f441e9-955f-4cdd-907a-c771e9968e90" containerName="main"
Jul 28 20:57:09.409385 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:09.409357    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="949574f9-d672-4509-9263-646f7df3613c" containerName="revoke-keys"
Jul 28 20:57:09.409385 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:09.409365    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="682a174e-e22f-44b0-9dfe-49ab56b00476" containerName="maas-api"
Jul 28 20:57:09.412548 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:09.412527    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-1ac855-zm67h"
Jul 28 20:57:09.415500 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:09.415472    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-982zh\""
Jul 28 20:57:09.421179 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:09.421152    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-1ac855-zm67h"]
Jul 28 20:57:09.609395 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:09.609352    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f7f4383-5717-4346-8a37-40402d3ff38b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-1ac855-zm67h\" (UID: \"5f7f4383-5717-4346-8a37-40402d3ff38b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-1ac855-zm67h"
Jul 28 20:57:09.710501 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:09.710419    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f7f4383-5717-4346-8a37-40402d3ff38b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-1ac855-zm67h\" (UID: \"5f7f4383-5717-4346-8a37-40402d3ff38b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-1ac855-zm67h"
Jul 28 20:57:09.711067 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:09.711031    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f7f4383-5717-4346-8a37-40402d3ff38b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-1ac855-zm67h\" (UID: \"5f7f4383-5717-4346-8a37-40402d3ff38b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-1ac855-zm67h"
Jul 28 20:57:09.724442 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:09.724416    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-1ac855-zm67h"
Jul 28 20:57:09.856798 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:09.856774    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-1ac855-zm67h"]
Jul 28 20:57:09.858895 ip-10-0-139-54 kubenswrapper[2554]: W0728 20:57:09.858865    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f7f4383_5717_4346_8a37_40402d3ff38b.slice/crio-3fb8717558b250d9fbcb0c62bf0a90541f558782736c7de7cddeb800b0e210a2 WatchSource:0}: Error finding container 3fb8717558b250d9fbcb0c62bf0a90541f558782736c7de7cddeb800b0e210a2: Status 404 returned error can't find the container with id 3fb8717558b250d9fbcb0c62bf0a90541f558782736c7de7cddeb800b0e210a2
Jul 28 20:57:10.675921 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:10.675827    2554 generic.go:356] "Generic (PLEG): container finished" podID="5f7f4383-5717-4346-8a37-40402d3ff38b" containerID="73a5aa84db9abd5d668260e170656b862786533c2b8fc7db1a84e37d3f0a1af3" exitCode=0
Jul 28 20:57:10.676331 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:10.675919    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-1ac855-zm67h" event={"ID":"5f7f4383-5717-4346-8a37-40402d3ff38b","Type":"ContainerDied","Data":"73a5aa84db9abd5d668260e170656b862786533c2b8fc7db1a84e37d3f0a1af3"}
Jul 28 20:57:10.676331 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:10.675983    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-1ac855-zm67h" event={"ID":"5f7f4383-5717-4346-8a37-40402d3ff38b","Type":"ContainerStarted","Data":"3fb8717558b250d9fbcb0c62bf0a90541f558782736c7de7cddeb800b0e210a2"}
Jul 28 20:57:11.809011 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:11.808987    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-1ac855-zm67h"
Jul 28 20:57:11.828357 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:11.828328    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f7f4383-5717-4346-8a37-40402d3ff38b-maas-api-service-ca\") pod \"5f7f4383-5717-4346-8a37-40402d3ff38b\" (UID: \"5f7f4383-5717-4346-8a37-40402d3ff38b\") "
Jul 28 20:57:11.828760 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:11.828735    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f7f4383-5717-4346-8a37-40402d3ff38b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5f7f4383-5717-4346-8a37-40402d3ff38b" (UID: "5f7f4383-5717-4346-8a37-40402d3ff38b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:57:11.929683 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:11.929646    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f7f4383-5717-4346-8a37-40402d3ff38b-maas-api-service-ca\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:57:12.685622 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:12.685590    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-1ac855-zm67h"
Jul 28 20:57:12.685622 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:12.685600    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-1ac855-zm67h" event={"ID":"5f7f4383-5717-4346-8a37-40402d3ff38b","Type":"ContainerDied","Data":"3fb8717558b250d9fbcb0c62bf0a90541f558782736c7de7cddeb800b0e210a2"}
Jul 28 20:57:12.685830 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:12.685644    2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3fb8717558b250d9fbcb0c62bf0a90541f558782736c7de7cddeb800b0e210a2"
Jul 28 20:57:15.416219 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:15.416184    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-1ac855-66d77475b8-66wbc"]
Jul 28 20:57:15.416723 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:15.416427    2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1ac855-66d77475b8-66wbc" podUID="d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff" containerName="maas-api" containerID="cri-o://3c446fdc03f71618b42ff863949c8ddcc7c5578d4ba72189931393075e72af99" gracePeriod=30
Jul 28 20:57:15.665506 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:15.665478    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1ac855-66d77475b8-66wbc"
Jul 28 20:57:15.701956 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:15.701874    2554 generic.go:356] "Generic (PLEG): container finished" podID="d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff" containerID="3c446fdc03f71618b42ff863949c8ddcc7c5578d4ba72189931393075e72af99" exitCode=0
Jul 28 20:57:15.701956 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:15.701918    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1ac855-66d77475b8-66wbc"
Jul 28 20:57:15.702181 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:15.701953    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1ac855-66d77475b8-66wbc" event={"ID":"d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff","Type":"ContainerDied","Data":"3c446fdc03f71618b42ff863949c8ddcc7c5578d4ba72189931393075e72af99"}
Jul 28 20:57:15.702181 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:15.702003    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1ac855-66d77475b8-66wbc" event={"ID":"d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff","Type":"ContainerDied","Data":"a1d143d696e4b4101cfc6fe23b589e32ed8c822910071d58a9f5aca645b6556c"}
Jul 28 20:57:15.702181 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:15.702028    2554 scope.go:117] "RemoveContainer" containerID="3c446fdc03f71618b42ff863949c8ddcc7c5578d4ba72189931393075e72af99"
Jul 28 20:57:15.711968 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:15.711947    2554 scope.go:117] "RemoveContainer" containerID="3c446fdc03f71618b42ff863949c8ddcc7c5578d4ba72189931393075e72af99"
Jul 28 20:57:15.712299 ip-10-0-139-54 kubenswrapper[2554]: E0728 20:57:15.712275    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c446fdc03f71618b42ff863949c8ddcc7c5578d4ba72189931393075e72af99\": container with ID starting with 3c446fdc03f71618b42ff863949c8ddcc7c5578d4ba72189931393075e72af99 not found: ID does not exist" containerID="3c446fdc03f71618b42ff863949c8ddcc7c5578d4ba72189931393075e72af99"
Jul 28 20:57:15.712407 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:15.712314    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c446fdc03f71618b42ff863949c8ddcc7c5578d4ba72189931393075e72af99"} err="failed to get container status \"3c446fdc03f71618b42ff863949c8ddcc7c5578d4ba72189931393075e72af99\": rpc error: code = NotFound desc = could not find container \"3c446fdc03f71618b42ff863949c8ddcc7c5578d4ba72189931393075e72af99\": container with ID starting with 3c446fdc03f71618b42ff863949c8ddcc7c5578d4ba72189931393075e72af99 not found: ID does not exist"
Jul 28 20:57:15.767522 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:15.767492    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff-openshift-service-ca\") pod \"d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff\" (UID: \"d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff\") "
Jul 28 20:57:15.767680 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:15.767536    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff-maas-api-tls\") pod \"d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff\" (UID: \"d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff\") "
Jul 28 20:57:15.767680 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:15.767623    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v9hzv\" (UniqueName: \"kubernetes.io/projected/d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff-kube-api-access-v9hzv\") pod \"d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff\" (UID: \"d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff\") "
Jul 28 20:57:15.767918 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:15.767889    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff" (UID: "d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:57:15.769687 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:15.769665    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff-kube-api-access-v9hzv" (OuterVolumeSpecName: "kube-api-access-v9hzv") pod "d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff" (UID: "d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff"). InnerVolumeSpecName "kube-api-access-v9hzv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:57:15.769756 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:15.769693    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff" (UID: "d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:57:15.868612 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:15.868579    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v9hzv\" (UniqueName: \"kubernetes.io/projected/d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff-kube-api-access-v9hzv\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:57:15.868612 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:15.868607    2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff-openshift-service-ca\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:57:15.868612 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:15.868619    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff-maas-api-tls\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:57:15.925867 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:15.925833    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-1ac855-zm67h"]
Jul 28 20:57:15.928488 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:15.928459    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-1ac855-zm67h"]
Jul 28 20:57:16.027717 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:16.027632    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-1ac855-66d77475b8-66wbc"]
Jul 28 20:57:16.035111 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:16.035057    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-1ac855-66d77475b8-66wbc"]
Jul 28 20:57:17.777905 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:17.777871    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f7f4383-5717-4346-8a37-40402d3ff38b" path="/var/lib/kubelet/pods/5f7f4383-5717-4346-8a37-40402d3ff38b/volumes"
Jul 28 20:57:17.778316 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:17.778287    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff" path="/var/lib/kubelet/pods/d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff/volumes"
Jul 28 20:57:23.134874 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:23.134777    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1b6922-6hcmk"]
Jul 28 20:57:23.135359 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:23.135336    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff" containerName="maas-api"
Jul 28 20:57:23.135432 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:23.135363    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff" containerName="maas-api"
Jul 28 20:57:23.135432 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:23.135377    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f7f4383-5717-4346-8a37-40402d3ff38b" containerName="revoke-keys"
Jul 28 20:57:23.135432 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:23.135385    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f7f4383-5717-4346-8a37-40402d3ff38b" containerName="revoke-keys"
Jul 28 20:57:23.135585 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:23.135485    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f7f4383-5717-4346-8a37-40402d3ff38b" containerName="revoke-keys"
Jul 28 20:57:23.135585 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:23.135503    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7799548-ae2a-4e1c-bcf4-5747b8d1d1ff" containerName="maas-api"
Jul 28 20:57:23.139036 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:23.139015    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1b6922-6hcmk"
Jul 28 20:57:23.142418 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:23.142399    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 20:57:23.142517 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:23.142402    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-982zh\""
Jul 28 20:57:23.148473 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:23.148447    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1b6922-6hcmk"]
Jul 28 20:57:23.233717 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:23.233685    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e75b0c6b-9e39-44be-8d63-88d64307cf96-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-1b6922-6hcmk\" (UID: \"e75b0c6b-9e39-44be-8d63-88d64307cf96\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1b6922-6hcmk"
Jul 28 20:57:23.334772 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:23.334731    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e75b0c6b-9e39-44be-8d63-88d64307cf96-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-1b6922-6hcmk\" (UID: \"e75b0c6b-9e39-44be-8d63-88d64307cf96\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1b6922-6hcmk"
Jul 28 20:57:23.335395 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:23.335371    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e75b0c6b-9e39-44be-8d63-88d64307cf96-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-1b6922-6hcmk\" (UID: \"e75b0c6b-9e39-44be-8d63-88d64307cf96\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1b6922-6hcmk"
Jul 28 20:57:23.451165 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:23.451061    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1b6922-6hcmk"
Jul 28 20:57:23.579145 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:23.579121    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1b6922-6hcmk"]
Jul 28 20:57:23.581890 ip-10-0-139-54 kubenswrapper[2554]: W0728 20:57:23.581847    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode75b0c6b_9e39_44be_8d63_88d64307cf96.slice/crio-a92872f7d06e0b42eb1200dbaa8c99935871bb116257710c3e1e944db22ab877 WatchSource:0}: Error finding container a92872f7d06e0b42eb1200dbaa8c99935871bb116257710c3e1e944db22ab877: Status 404 returned error can't find the container with id a92872f7d06e0b42eb1200dbaa8c99935871bb116257710c3e1e944db22ab877
Jul 28 20:57:23.746683 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:23.746599    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1b6922-6hcmk" event={"ID":"e75b0c6b-9e39-44be-8d63-88d64307cf96","Type":"ContainerStarted","Data":"a92872f7d06e0b42eb1200dbaa8c99935871bb116257710c3e1e944db22ab877"}
Jul 28 20:57:24.752623 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:24.752536    2554 generic.go:356] "Generic (PLEG): container finished" podID="e75b0c6b-9e39-44be-8d63-88d64307cf96" containerID="defbb6d2acc3b99b24ba6b42980090a96cfe2839d4ebc50c1cbc55da67beda1e" exitCode=0
Jul 28 20:57:24.752995 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:24.752629    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1b6922-6hcmk" event={"ID":"e75b0c6b-9e39-44be-8d63-88d64307cf96","Type":"ContainerDied","Data":"defbb6d2acc3b99b24ba6b42980090a96cfe2839d4ebc50c1cbc55da67beda1e"}
Jul 28 20:57:25.884209 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:25.884181    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1b6922-6hcmk"
Jul 28 20:57:25.957195 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:25.957156    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e75b0c6b-9e39-44be-8d63-88d64307cf96-maas-api-service-ca\") pod \"e75b0c6b-9e39-44be-8d63-88d64307cf96\" (UID: \"e75b0c6b-9e39-44be-8d63-88d64307cf96\") "
Jul 28 20:57:25.957523 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:25.957500    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e75b0c6b-9e39-44be-8d63-88d64307cf96-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e75b0c6b-9e39-44be-8d63-88d64307cf96" (UID: "e75b0c6b-9e39-44be-8d63-88d64307cf96"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:57:26.058043 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:26.058006    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e75b0c6b-9e39-44be-8d63-88d64307cf96-maas-api-service-ca\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:57:26.764617 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:26.764575    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1b6922-6hcmk" event={"ID":"e75b0c6b-9e39-44be-8d63-88d64307cf96","Type":"ContainerDied","Data":"a92872f7d06e0b42eb1200dbaa8c99935871bb116257710c3e1e944db22ab877"}
Jul 28 20:57:26.764790 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:26.764658    2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a92872f7d06e0b42eb1200dbaa8c99935871bb116257710c3e1e944db22ab877"
Jul 28 20:57:26.764790 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:26.764595    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1b6922-6hcmk"
Jul 28 20:57:30.866270 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:30.866229    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1b6922-6hcmk"]
Jul 28 20:57:30.868923 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:30.868894    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-1b6922-6hcmk"]
Jul 28 20:57:31.777999 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:31.777968    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e75b0c6b-9e39-44be-8d63-88d64307cf96" path="/var/lib/kubelet/pods/e75b0c6b-9e39-44be-8d63-88d64307cf96/volumes"
Jul 28 20:57:38.468861 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:38.468829    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c0e826-smm66"]
Jul 28 20:57:38.469258 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:38.469180    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e75b0c6b-9e39-44be-8d63-88d64307cf96" containerName="revoke-keys"
Jul 28 20:57:38.469258 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:38.469191    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="e75b0c6b-9e39-44be-8d63-88d64307cf96" containerName="revoke-keys"
Jul 28 20:57:38.469331 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:38.469265    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="e75b0c6b-9e39-44be-8d63-88d64307cf96" containerName="revoke-keys"
Jul 28 20:57:38.471410 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:38.471394    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c0e826-smm66"
Jul 28 20:57:38.474243 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:38.474221    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 20:57:38.474444 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:38.474431    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-982zh\""
Jul 28 20:57:38.481530 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:38.481505    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c0e826-smm66"]
Jul 28 20:57:38.581104 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:38.581036    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c006a8b-9d81-44a5-a15a-2997b96bebee-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-c0e826-smm66\" (UID: \"4c006a8b-9d81-44a5-a15a-2997b96bebee\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c0e826-smm66"
Jul 28 20:57:38.681735 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:38.681699    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c006a8b-9d81-44a5-a15a-2997b96bebee-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-c0e826-smm66\" (UID: \"4c006a8b-9d81-44a5-a15a-2997b96bebee\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c0e826-smm66"
Jul 28 20:57:38.682354 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:38.682333    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c006a8b-9d81-44a5-a15a-2997b96bebee-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-c0e826-smm66\" (UID: \"4c006a8b-9d81-44a5-a15a-2997b96bebee\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c0e826-smm66"
Jul 28 20:57:38.782973 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:38.782879    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c0e826-smm66"
Jul 28 20:57:38.921261 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:38.921236    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c0e826-smm66"]
Jul 28 20:57:38.923629 ip-10-0-139-54 kubenswrapper[2554]: W0728 20:57:38.923593    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c006a8b_9d81_44a5_a15a_2997b96bebee.slice/crio-3409b37e1745d477093609c88df9c51008420ebc3b3eaa1600ac6aaa1fd8c6e5 WatchSource:0}: Error finding container 3409b37e1745d477093609c88df9c51008420ebc3b3eaa1600ac6aaa1fd8c6e5: Status 404 returned error can't find the container with id 3409b37e1745d477093609c88df9c51008420ebc3b3eaa1600ac6aaa1fd8c6e5
Jul 28 20:57:39.836001 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:39.835959    2554 generic.go:356] "Generic (PLEG): container finished" podID="4c006a8b-9d81-44a5-a15a-2997b96bebee" containerID="1a5360e5d509ca0f85a8342fafa22c4aaf8f6d00bd88d3cda8b446721dee5dc6" exitCode=0
Jul 28 20:57:39.836001 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:39.836004    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c0e826-smm66" event={"ID":"4c006a8b-9d81-44a5-a15a-2997b96bebee","Type":"ContainerDied","Data":"1a5360e5d509ca0f85a8342fafa22c4aaf8f6d00bd88d3cda8b446721dee5dc6"}
Jul 28 20:57:39.836448 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:39.836084    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c0e826-smm66" event={"ID":"4c006a8b-9d81-44a5-a15a-2997b96bebee","Type":"ContainerStarted","Data":"3409b37e1745d477093609c88df9c51008420ebc3b3eaa1600ac6aaa1fd8c6e5"}
Jul 28 20:57:40.973586 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:40.973563    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c0e826-smm66"
Jul 28 20:57:41.105389 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:41.105310    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c006a8b-9d81-44a5-a15a-2997b96bebee-maas-api-service-ca\") pod \"4c006a8b-9d81-44a5-a15a-2997b96bebee\" (UID: \"4c006a8b-9d81-44a5-a15a-2997b96bebee\") "
Jul 28 20:57:41.105713 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:41.105683    2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c006a8b-9d81-44a5-a15a-2997b96bebee-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4c006a8b-9d81-44a5-a15a-2997b96bebee" (UID: "4c006a8b-9d81-44a5-a15a-2997b96bebee"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:57:41.206218 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:41.206179    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c006a8b-9d81-44a5-a15a-2997b96bebee-maas-api-service-ca\") on node \"ip-10-0-139-54.ec2.internal\" DevicePath \"\""
Jul 28 20:57:41.849207 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:41.849175    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c0e826-smm66"
Jul 28 20:57:41.849387 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:41.849212    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c0e826-smm66" event={"ID":"4c006a8b-9d81-44a5-a15a-2997b96bebee","Type":"ContainerDied","Data":"3409b37e1745d477093609c88df9c51008420ebc3b3eaa1600ac6aaa1fd8c6e5"}
Jul 28 20:57:41.849387 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:41.849257    2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3409b37e1745d477093609c88df9c51008420ebc3b3eaa1600ac6aaa1fd8c6e5"
Jul 28 20:57:43.856892 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:43.856859    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c0e826-smm66"]
Jul 28 20:57:43.860698 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:43.860668    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c0e826-smm66"]
Jul 28 20:57:45.777390 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:45.777360    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c006a8b-9d81-44a5-a15a-2997b96bebee" path="/var/lib/kubelet/pods/4c006a8b-9d81-44a5-a15a-2997b96bebee/volumes"
Jul 28 20:57:57.741052 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:57.741025    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-685bc94585-57q9x_86591ebe-d149-4205-9b93-0bfa8bc94e72/copy-manifests/0.log"
Jul 28 20:57:57.748426 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:57:57.748398    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-685bc94585-57q9x_86591ebe-d149-4205-9b93-0bfa8bc94e72/manager/0.log"
Jul 28 20:58:00.455911 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:00.455880    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-frxgw_30697c07-d9ef-4436-93b0-e7f140368a1d/manager/0.log"
Jul 28 20:58:00.567951 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:00.567922    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-gl8p6_7b5bc3d2-d2f5-4944-bcb0-f45ab40c4dc0/kuadrant-console-plugin/0.log"
Jul 28 20:58:01.056592 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:01.056555    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-vp6pt_cbedc35d-405b-421c-ba08-f7051525b560/manager/0.log"
Jul 28 20:58:01.823127 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:01.823091    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6cdf7b948d-wtcxn_44069363-b129-408c-90ce-a4c92b46d1b9/kube-auth-proxy/0.log"
Jul 28 20:58:02.310629 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:02.310603    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7c5b95cbd9-xk69c_bb24870f-3c94-4836-9e8b-8d6f757c84de/router/0.log"
Jul 28 20:58:02.781625 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:02.781521    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-zx8p7_09cc7986-c2a0-49d2-8214-0abfd81d6b8a/main/0.log"
Jul 28 20:58:02.903476 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:02.903450    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-49sbr_8131278e-1e59-4a56-83a2-ed9a0b882c98/main/0.log"
Jul 28 20:58:03.139820 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:03.139792    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-ktsv2_c9d3df2d-be95-44cd-8d58-6f93fd9ed81b/main/0.log"
Jul 28 20:58:03.252138 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:03.252102    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-tvqhc_e29b582c-3dc7-4e0e-af68-83f39cbbfc0d/main/0.log"
Jul 28 20:58:12.345182 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:12.345142    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-rv7fr_9cc5727b-a071-4793-a0b1-b92f6287080a/global-pull-secret-syncer/0.log"
Jul 28 20:58:12.494064 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:12.494021    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-xs5fh_acc81073-b397-4ed9-8e21-5826814e303e/konnectivity-agent/0.log"
Jul 28 20:58:12.619439 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:12.619353    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-54.ec2.internal_74845271ca622f52b49ec9ab9ec8f7b9/haproxy/0.log"
Jul 28 20:58:17.454086 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:17.454045    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-frxgw_30697c07-d9ef-4436-93b0-e7f140368a1d/manager/0.log"
Jul 28 20:58:17.483308 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:17.483279    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-gl8p6_7b5bc3d2-d2f5-4944-bcb0-f45ab40c4dc0/kuadrant-console-plugin/0.log"
Jul 28 20:58:17.905370 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:17.905333    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-vp6pt_cbedc35d-405b-421c-ba08-f7051525b560/manager/0.log"
Jul 28 20:58:19.830113 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:19.830049    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-5bj5s_15900109-73a5-4356-b125-6b73b7743570/cluster-monitoring-operator/0.log"
Jul 28 20:58:20.216778 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:20.216735    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pv6xj_cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd/node-exporter/0.log"
Jul 28 20:58:20.262210 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:20.262176    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pv6xj_cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd/kube-rbac-proxy/0.log"
Jul 28 20:58:20.307227 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:20.307204    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pv6xj_cb6b59a5-2a48-4d15-8dd9-4c19242bc9cd/init-textfile/0.log"
Jul 28 20:58:20.457665 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:20.457572    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_60850744-b39b-4b11-af79-d5baac9854c0/prometheus/0.log"
Jul 28 20:58:20.481325 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:20.481292    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_60850744-b39b-4b11-af79-d5baac9854c0/config-reloader/0.log"
Jul 28 20:58:20.509242 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:20.509214    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_60850744-b39b-4b11-af79-d5baac9854c0/thanos-sidecar/0.log"
Jul 28 20:58:20.534509 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:20.534483    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_60850744-b39b-4b11-af79-d5baac9854c0/kube-rbac-proxy-web/0.log"
Jul 28 20:58:20.558545 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:20.558515    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_60850744-b39b-4b11-af79-d5baac9854c0/kube-rbac-proxy/0.log"
Jul 28 20:58:20.580964 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:20.580921    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_60850744-b39b-4b11-af79-d5baac9854c0/kube-rbac-proxy-thanos/0.log"
Jul 28 20:58:20.603933 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:20.603910    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_60850744-b39b-4b11-af79-d5baac9854c0/init-config-reloader/0.log"
Jul 28 20:58:22.224929 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:22.224883    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-kdk6l/perf-node-gather-daemonset-fldl8"]
Jul 28 20:58:22.225331 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:22.225243    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c006a8b-9d81-44a5-a15a-2997b96bebee" containerName="revoke-keys"
Jul 28 20:58:22.225331 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:22.225255    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c006a8b-9d81-44a5-a15a-2997b96bebee" containerName="revoke-keys"
Jul 28 20:58:22.225331 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:22.225328    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c006a8b-9d81-44a5-a15a-2997b96bebee" containerName="revoke-keys"
Jul 28 20:58:22.227462 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:22.227446    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kdk6l/perf-node-gather-daemonset-fldl8"
Jul 28 20:58:22.230492 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:22.230470    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-kdk6l\"/\"default-dockercfg-cqn65\""
Jul 28 20:58:22.230690 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:22.230677    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kdk6l\"/\"openshift-service-ca.crt\""
Jul 28 20:58:22.231734 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:22.231721    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kdk6l\"/\"kube-root-ca.crt\""
Jul 28 20:58:22.239676 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:22.239656    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kdk6l/perf-node-gather-daemonset-fldl8"]
Jul 28 20:58:22.362435 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:22.362396    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shqkx\" (UniqueName: \"kubernetes.io/projected/05711e9d-b464-49b2-9457-3d1080d7c877-kube-api-access-shqkx\") pod \"perf-node-gather-daemonset-fldl8\" (UID: \"05711e9d-b464-49b2-9457-3d1080d7c877\") " pod="openshift-must-gather-kdk6l/perf-node-gather-daemonset-fldl8"
Jul 28 20:58:22.362435 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:22.362437    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/05711e9d-b464-49b2-9457-3d1080d7c877-lib-modules\") pod \"perf-node-gather-daemonset-fldl8\" (UID: \"05711e9d-b464-49b2-9457-3d1080d7c877\") " pod="openshift-must-gather-kdk6l/perf-node-gather-daemonset-fldl8"
Jul 28 20:58:22.362664 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:22.362462    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/05711e9d-b464-49b2-9457-3d1080d7c877-sys\") pod \"perf-node-gather-daemonset-fldl8\" (UID: \"05711e9d-b464-49b2-9457-3d1080d7c877\") " pod="openshift-must-gather-kdk6l/perf-node-gather-daemonset-fldl8"
Jul 28 20:58:22.362664 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:22.362538    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/05711e9d-b464-49b2-9457-3d1080d7c877-podres\") pod \"perf-node-gather-daemonset-fldl8\" (UID: \"05711e9d-b464-49b2-9457-3d1080d7c877\") " pod="openshift-must-gather-kdk6l/perf-node-gather-daemonset-fldl8"
Jul 28 20:58:22.362664 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:22.362636    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/05711e9d-b464-49b2-9457-3d1080d7c877-proc\") pod \"perf-node-gather-daemonset-fldl8\" (UID: \"05711e9d-b464-49b2-9457-3d1080d7c877\") " pod="openshift-must-gather-kdk6l/perf-node-gather-daemonset-fldl8"
Jul 28 20:58:22.463498 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:22.463457    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/05711e9d-b464-49b2-9457-3d1080d7c877-proc\") pod \"perf-node-gather-daemonset-fldl8\" (UID: \"05711e9d-b464-49b2-9457-3d1080d7c877\") " pod="openshift-must-gather-kdk6l/perf-node-gather-daemonset-fldl8"
Jul 28 20:58:22.463665 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:22.463508    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shqkx\" (UniqueName: \"kubernetes.io/projected/05711e9d-b464-49b2-9457-3d1080d7c877-kube-api-access-shqkx\") pod \"perf-node-gather-daemonset-fldl8\" (UID: \"05711e9d-b464-49b2-9457-3d1080d7c877\") " pod="openshift-must-gather-kdk6l/perf-node-gather-daemonset-fldl8"
Jul 28 20:58:22.463665 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:22.463528    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/05711e9d-b464-49b2-9457-3d1080d7c877-lib-modules\") pod \"perf-node-gather-daemonset-fldl8\" (UID: \"05711e9d-b464-49b2-9457-3d1080d7c877\") " pod="openshift-must-gather-kdk6l/perf-node-gather-daemonset-fldl8"
Jul 28 20:58:22.463665 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:22.463561    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/05711e9d-b464-49b2-9457-3d1080d7c877-sys\") pod \"perf-node-gather-daemonset-fldl8\" (UID: \"05711e9d-b464-49b2-9457-3d1080d7c877\") " pod="openshift-must-gather-kdk6l/perf-node-gather-daemonset-fldl8"
Jul 28 20:58:22.463665 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:22.463584    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/05711e9d-b464-49b2-9457-3d1080d7c877-proc\") pod \"perf-node-gather-daemonset-fldl8\" (UID: \"05711e9d-b464-49b2-9457-3d1080d7c877\") " pod="openshift-must-gather-kdk6l/perf-node-gather-daemonset-fldl8"
Jul 28 20:58:22.463665 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:22.463591    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/05711e9d-b464-49b2-9457-3d1080d7c877-podres\") pod \"perf-node-gather-daemonset-fldl8\" (UID: \"05711e9d-b464-49b2-9457-3d1080d7c877\") " pod="openshift-must-gather-kdk6l/perf-node-gather-daemonset-fldl8"
Jul 28 20:58:22.463841 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:22.463688    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/05711e9d-b464-49b2-9457-3d1080d7c877-sys\") pod \"perf-node-gather-daemonset-fldl8\" (UID: \"05711e9d-b464-49b2-9457-3d1080d7c877\") " pod="openshift-must-gather-kdk6l/perf-node-gather-daemonset-fldl8"
Jul 28 20:58:22.463841 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:22.463702    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/05711e9d-b464-49b2-9457-3d1080d7c877-podres\") pod \"perf-node-gather-daemonset-fldl8\" (UID: \"05711e9d-b464-49b2-9457-3d1080d7c877\") " pod="openshift-must-gather-kdk6l/perf-node-gather-daemonset-fldl8"
Jul 28 20:58:22.463841 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:22.463703    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/05711e9d-b464-49b2-9457-3d1080d7c877-lib-modules\") pod \"perf-node-gather-daemonset-fldl8\" (UID: \"05711e9d-b464-49b2-9457-3d1080d7c877\") " pod="openshift-must-gather-kdk6l/perf-node-gather-daemonset-fldl8"
Jul 28 20:58:22.473199 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:22.473173    2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shqkx\" (UniqueName: \"kubernetes.io/projected/05711e9d-b464-49b2-9457-3d1080d7c877-kube-api-access-shqkx\") pod \"perf-node-gather-daemonset-fldl8\" (UID: \"05711e9d-b464-49b2-9457-3d1080d7c877\") " pod="openshift-must-gather-kdk6l/perf-node-gather-daemonset-fldl8"
Jul 28 20:58:22.537431 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:22.537403    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kdk6l/perf-node-gather-daemonset-fldl8"
Jul 28 20:58:22.663173 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:22.663104    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kdk6l/perf-node-gather-daemonset-fldl8"]
Jul 28 20:58:22.665503 ip-10-0-139-54 kubenswrapper[2554]: W0728 20:58:22.665471    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod05711e9d_b464_49b2_9457_3d1080d7c877.slice/crio-1b968d6e42a17f4958abfa0fa7f0c118480df2e72e4b2f58c95b2738953a4678 WatchSource:0}: Error finding container 1b968d6e42a17f4958abfa0fa7f0c118480df2e72e4b2f58c95b2738953a4678: Status 404 returned error can't find the container with id 1b968d6e42a17f4958abfa0fa7f0c118480df2e72e4b2f58c95b2738953a4678
Jul 28 20:58:22.801903 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:22.801811    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-7vcr8_dde4c1a8-8229-4419-87b6-d8e39a0827c4/console-operator/1.log"
Jul 28 20:58:22.809668 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:22.809637    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-7vcr8_dde4c1a8-8229-4419-87b6-d8e39a0827c4/console-operator/2.log"
Jul 28 20:58:23.049917 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:23.049883    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kdk6l/perf-node-gather-daemonset-fldl8" event={"ID":"05711e9d-b464-49b2-9457-3d1080d7c877","Type":"ContainerStarted","Data":"98a27d61c471be7943ad1d9ff468798ca0645bafa9201a7d1ea873a437aa5222"}
Jul 28 20:58:23.050097 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:23.049957    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kdk6l/perf-node-gather-daemonset-fldl8" event={"ID":"05711e9d-b464-49b2-9457-3d1080d7c877","Type":"ContainerStarted","Data":"1b968d6e42a17f4958abfa0fa7f0c118480df2e72e4b2f58c95b2738953a4678"}
Jul 28 20:58:23.050097 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:23.050024    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-kdk6l/perf-node-gather-daemonset-fldl8"
Jul 28 20:58:23.068583 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:23.068499    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-kdk6l/perf-node-gather-daemonset-fldl8" podStartSLOduration=1.068487999 podStartE2EDuration="1.068487999s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 20:58:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:58:23.067302979 +0000 UTC m=+3565.850314433" watchObservedRunningTime="2026-07-28 20:58:23.068487999 +0000 UTC m=+3565.851499440"
Jul 28 20:58:23.399782 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:23.399701    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-zdppx_780efb21-e22d-46d1-b7c2-4ab44c7f4600/download-server/0.log"
Jul 28 20:58:24.119473 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:24.119442    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-bqs2c_454f1f68-9403-4c21-ba51-55536c8f55c9/volume-data-source-validator/0.log"
Jul 28 20:58:25.233154 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:25.233122    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qs4pv_2b7b89ba-b64b-454a-b032-34757974c00a/dns/0.log"
Jul 28 20:58:25.257812 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:25.257786    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qs4pv_2b7b89ba-b64b-454a-b032-34757974c00a/kube-rbac-proxy/0.log"
Jul 28 20:58:25.331710 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:25.331679    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-42pjq_c254f770-a33c-4a75-bd75-9bdc3e4e12ed/dns-node-resolver/0.log"
Jul 28 20:58:26.068228 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:26.068177    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qrztd_38cf5bd2-15db-49bc-b41d-b0df3ea06cb1/node-ca/0.log"
Jul 28 20:58:27.299749 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:27.299720    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6cdf7b948d-wtcxn_44069363-b129-408c-90ce-a4c92b46d1b9/kube-auth-proxy/0.log"
Jul 28 20:58:27.488541 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:27.488505    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7c5b95cbd9-xk69c_bb24870f-3c94-4836-9e8b-8d6f757c84de/router/0.log"
Jul 28 20:58:28.401635 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:28.401595    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xg646_e69543ef-79c9-4a8f-bab6-8e1af517c029/serve-healthcheck-canary/0.log"
Jul 28 20:58:28.910344 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:28.910308    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-mfvfs_26b203ca-0b58-4389-ac85-8fdc909e11d2/insights-operator/0.log"
Jul 28 20:58:28.912064 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:28.912041    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-mfvfs_26b203ca-0b58-4389-ac85-8fdc909e11d2/insights-operator/1.log"
Jul 28 20:58:29.065456 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:29.065430    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-kdk6l/perf-node-gather-daemonset-fldl8"
Jul 28 20:58:29.100915 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:29.100890    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-trhgs_be39a539-0c2c-4e08-9d85-c5ade81491a4/kube-rbac-proxy/0.log"
Jul 28 20:58:29.128831 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:29.128802    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-trhgs_be39a539-0c2c-4e08-9d85-c5ade81491a4/exporter/0.log"
Jul 28 20:58:29.154700 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:29.154673    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-trhgs_be39a539-0c2c-4e08-9d85-c5ade81491a4/extractor/0.log"
Jul 28 20:58:32.225558 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:32.225529    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-685bc94585-57q9x_86591ebe-d149-4205-9b93-0bfa8bc94e72/manager/0.log"
Jul 28 20:58:32.251178 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:32.251155    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-685bc94585-57q9x_86591ebe-d149-4205-9b93-0bfa8bc94e72/copy-manifests/0.log"
Jul 28 20:58:33.136333 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:33.136305    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6cdf7b948d-wtcxn_44069363-b129-408c-90ce-a4c92b46d1b9/kube-auth-proxy/0.log"
Jul 28 20:58:33.250459 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:33.250422    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7c5b95cbd9-xk69c_bb24870f-3c94-4836-9e8b-8d6f757c84de/router/0.log"
Jul 28 20:58:33.705805 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:33.705780    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-789bdb4f85-rfnwn_ec7bd9e6-90c1-468b-ba24-d274f070e2b9/manager/0.log"
Jul 28 20:58:34.823171 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:34.823136    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6cdf7b948d-wtcxn_44069363-b129-408c-90ce-a4c92b46d1b9/kube-auth-proxy/0.log"
Jul 28 20:58:34.951641 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:34.951610    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7c5b95cbd9-xk69c_bb24870f-3c94-4836-9e8b-8d6f757c84de/router/0.log"
Jul 28 20:58:39.052356 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:39.052319    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-k9b95_8f22d663-ad11-402a-a2a6-2f66e778b4ff/kube-storage-version-migrator-operator/1.log"
Jul 28 20:58:39.054032 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:39.054008    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-k9b95_8f22d663-ad11-402a-a2a6-2f66e778b4ff/kube-storage-version-migrator-operator/0.log"
Jul 28 20:58:40.375882 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:40.375855    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7p48k_ea4abf49-55cf-4351-a952-7b7a83e803d9/kube-multus-additional-cni-plugins/0.log"
Jul 28 20:58:40.399147 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:40.399112    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7p48k_ea4abf49-55cf-4351-a952-7b7a83e803d9/egress-router-binary-copy/0.log"
Jul 28 20:58:40.428393 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:40.428366    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7p48k_ea4abf49-55cf-4351-a952-7b7a83e803d9/cni-plugins/0.log"
Jul 28 20:58:40.457583 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:40.457556    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7p48k_ea4abf49-55cf-4351-a952-7b7a83e803d9/bond-cni-plugin/0.log"
Jul 28 20:58:40.480600 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:40.480573    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7p48k_ea4abf49-55cf-4351-a952-7b7a83e803d9/routeoverride-cni/0.log"
Jul 28 20:58:40.503787 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:40.503763    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7p48k_ea4abf49-55cf-4351-a952-7b7a83e803d9/whereabouts-cni-bincopy/0.log"
Jul 28 20:58:40.532944 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:40.532917    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7p48k_ea4abf49-55cf-4351-a952-7b7a83e803d9/whereabouts-cni/0.log"
Jul 28 20:58:40.793862 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:40.793816    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-xxgzp_6485753f-f774-4aa4-aaec-130c7b7ba868/kube-multus/0.log"
Jul 28 20:58:40.864001 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:40.863970    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-xmd9l_752faedf-0642-47aa-88bb-72f0ae961ebb/network-metrics-daemon/0.log"
Jul 28 20:58:40.886935 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:40.886910    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-xmd9l_752faedf-0642-47aa-88bb-72f0ae961ebb/kube-rbac-proxy/0.log"
Jul 28 20:58:42.040672 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:42.040643    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qxw76_f8bb4b77-7948-42bb-9433-d048c7205a61/ovn-controller/0.log"
Jul 28 20:58:42.059495 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:42.059467    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qxw76_f8bb4b77-7948-42bb-9433-d048c7205a61/ovn-acl-logging/0.log"
Jul 28 20:58:42.087873 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:42.087847    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qxw76_f8bb4b77-7948-42bb-9433-d048c7205a61/kube-rbac-proxy-node/0.log"
Jul 28 20:58:42.109945 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:42.109914    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qxw76_f8bb4b77-7948-42bb-9433-d048c7205a61/kube-rbac-proxy-ovn-metrics/0.log"
Jul 28 20:58:42.138657 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:42.138627    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qxw76_f8bb4b77-7948-42bb-9433-d048c7205a61/northd/0.log"
Jul 28 20:58:42.184732 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:42.184700    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qxw76_f8bb4b77-7948-42bb-9433-d048c7205a61/nbdb/0.log"
Jul 28 20:58:42.250406 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:42.250379    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qxw76_f8bb4b77-7948-42bb-9433-d048c7205a61/sbdb/0.log"
Jul 28 20:58:42.348023 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:42.347940    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qxw76_f8bb4b77-7948-42bb-9433-d048c7205a61/ovnkube-controller/0.log"
Jul 28 20:58:43.671265 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:43.671226    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-7d88cb55d8-72w9l_d0d19ca1-a117-47c3-a19b-eb756ce208cf/check-endpoints/0.log"
Jul 28 20:58:43.729365 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:43.729337    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-v2g2z_2f157e54-c70e-46b1-a41b-d60279cd7189/network-check-target-container/0.log"
Jul 28 20:58:44.841248 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:44.841214    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-9qlvt_03099009-562b-4d4b-9db8-95ec824e4b1c/iptables-alerter/0.log"
Jul 28 20:58:45.579767 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:45.579736    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-9lw2g_95c98833-0ec7-45eb-8986-14d40c9986e8/tuned/0.log"
Jul 28 20:58:47.828399 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:47.828367    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-6748489656-p4m9x_689af28a-582d-4dfe-a7f9-37cad631f34a/cluster-samples-operator/0.log"
Jul 28 20:58:47.846868 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:47.846834    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-6748489656-p4m9x_689af28a-582d-4dfe-a7f9-37cad631f34a/cluster-samples-operator-watch/0.log"
Jul 28 20:58:48.894673 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:48.894641    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-service-ca-operator_service-ca-operator-5d6c75f84f-xjfzb_2d7a85b0-e81f-4686-8889-5096efe82a1a/service-ca-operator/1.log"
Jul 28 20:58:48.896184 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:48.896153    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-service-ca-operator_service-ca-operator-5d6c75f84f-xjfzb_2d7a85b0-e81f-4686-8889-5096efe82a1a/service-ca-operator/0.log"
Jul 28 20:58:49.911135 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:49.911035    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-w779p_746a745b-0cf6-4b92-8173-4a8932ec5dff/csi-driver/0.log"
Jul 28 20:58:49.935251 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:49.935220    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-w779p_746a745b-0cf6-4b92-8173-4a8932ec5dff/csi-node-driver-registrar/0.log"
Jul 28 20:58:49.957039 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:49.957015    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-w779p_746a745b-0cf6-4b92-8173-4a8932ec5dff/csi-liveness-probe/0.log"
Jul 28 20:58:58.010199 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:58.010086    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-7vcr8_dde4c1a8-8229-4419-87b6-d8e39a0827c4/console-operator/1.log"
Jul 28 20:58:58.015240 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:58:58.014063    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-7vcr8_dde4c1a8-8229-4419-87b6-d8e39a0827c4/console-operator/1.log"
Jul 28 20:59:58.124636 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:59:58.124607    2554 scope.go:117] "RemoveContainer" containerID="3a68ad79ecc9656e18335deb83eabb4bd6175450f07940c612155e956302eb25"
Jul 28 20:59:58.132749 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:59:58.132728    2554 scope.go:117] "RemoveContainer" containerID="c7b7a0956eb7b24ab10e936dfdd48b634501746b1c79052262e9f0f33d94141b"
Jul 28 20:59:58.140152 ip-10-0-139-54 kubenswrapper[2554]: I0728 20:59:58.140134    2554 scope.go:117] "RemoveContainer" containerID="23b931796725d5e20981b1f4493697f01d33ae1793ab9ce56cf01148ed5f799c"