Jul 07 17:05:43.848460 ip-10-0-142-103 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 07 17:05:43.848470 ip-10-0-142-103 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 07 17:05:43.848477 ip-10-0-142-103 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 07 17:05:43.848698 ip-10-0-142-103 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 07 17:05:53.888119 ip-10-0-142-103 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 07 17:05:53.888138 ip-10-0-142-103 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 7b9f3cc74f43487ea9934a681e0acbec --
Jul 07 17:07:56.389227 ip-10-0-142-103 systemd[1]: Starting Kubernetes Kubelet...
Jul 07 17:07:56.771647 ip-10-0-142-103 kubenswrapper[2579]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 17:07:56.771647 ip-10-0-142-103 kubenswrapper[2579]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 07 17:07:56.771647 ip-10-0-142-103 kubenswrapper[2579]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 17:07:56.771647 ip-10-0-142-103 kubenswrapper[2579]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 07 17:07:56.771647 ip-10-0-142-103 kubenswrapper[2579]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 17:07:56.772703 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.772607    2579 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 07 17:07:56.776271 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776250    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 17:07:56.776271 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776270    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 17:07:56.776343 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776275    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 17:07:56.776343 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776278    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 17:07:56.776343 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776281    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 17:07:56.776343 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776285    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 17:07:56.776343 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776288    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 17:07:56.776343 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776291    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 17:07:56.776343 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776294    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 17:07:56.776343 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776297    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 17:07:56.776343 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776300    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 17:07:56.776343 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776303    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 17:07:56.776343 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776305    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 17:07:56.776343 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776308    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 17:07:56.776343 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776311    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 17:07:56.776343 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776321    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 17:07:56.776343 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776324    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 17:07:56.776343 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776327    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 17:07:56.776343 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776331    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 17:07:56.776343 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776335    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 17:07:56.776343 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776338    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 17:07:56.776343 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776341    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 17:07:56.776939 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776344    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 17:07:56.776939 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776347    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 17:07:56.776939 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776350    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 17:07:56.776939 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776353    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 17:07:56.776939 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776356    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 17:07:56.776939 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776359    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 17:07:56.776939 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776361    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 17:07:56.776939 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776364    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 17:07:56.776939 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776366    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 17:07:56.776939 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776369    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 17:07:56.776939 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776371    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 17:07:56.776939 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776374    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 17:07:56.776939 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776376    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 17:07:56.776939 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776379    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 17:07:56.776939 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776381    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 17:07:56.776939 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776384    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 17:07:56.776939 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776386    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 17:07:56.776939 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776389    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 17:07:56.776939 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776391    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 17:07:56.777402 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776394    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 17:07:56.777402 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776396    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 17:07:56.777402 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776399    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 17:07:56.777402 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776402    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 17:07:56.777402 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776405    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 17:07:56.777402 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776407    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 17:07:56.777402 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776410    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 17:07:56.777402 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776412    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 17:07:56.777402 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776415    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 17:07:56.777402 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776417    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 17:07:56.777402 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776419    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 17:07:56.777402 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776421    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 17:07:56.777402 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776424    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 17:07:56.777402 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776426    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 17:07:56.777402 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776429    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 17:07:56.777402 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776432    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 17:07:56.777402 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776434    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 17:07:56.777402 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776438    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 17:07:56.777402 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776441    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 17:07:56.777402 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776443    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 17:07:56.777909 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776446    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 17:07:56.777909 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776448    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 17:07:56.777909 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776450    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 17:07:56.777909 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776453    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 17:07:56.777909 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776456    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 17:07:56.777909 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776458    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 17:07:56.777909 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776461    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 17:07:56.777909 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776463    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 17:07:56.777909 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776467    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 17:07:56.777909 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776471    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 17:07:56.777909 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776473    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 17:07:56.777909 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776475    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 17:07:56.777909 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776478    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 17:07:56.777909 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776480    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 17:07:56.777909 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776483    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 17:07:56.777909 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776485    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 17:07:56.777909 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776488    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 17:07:56.777909 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776491    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 17:07:56.777909 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776493    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 17:07:56.777909 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776496    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 17:07:56.778393 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776498    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 17:07:56.778393 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776500    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 17:07:56.778393 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776503    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 17:07:56.778393 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776508    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 17:07:56.778393 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776511    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 17:07:56.778393 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776515    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 17:07:56.778393 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776519    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 17:07:56.778393 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776522    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 17:07:56.778393 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776525    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 17:07:56.778393 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776527    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 17:07:56.778393 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776530    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 17:07:56.778393 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776533    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 17:07:56.778393 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776536    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 17:07:56.778393 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776538    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 17:07:56.778393 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776542    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 17:07:56.778393 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776545    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 17:07:56.778393 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776547    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 17:07:56.778393 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.776549    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 17:07:56.778393 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777292    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 17:07:56.778393 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777300    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 17:07:56.778877 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777303    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 17:07:56.778877 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777306    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 17:07:56.778877 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777309    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 17:07:56.778877 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777312    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 17:07:56.778877 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777315    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 17:07:56.778877 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777317    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 17:07:56.778877 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777320    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 17:07:56.778877 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777323    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 17:07:56.778877 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777325    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 17:07:56.778877 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777328    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 17:07:56.778877 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777331    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 17:07:56.778877 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777333    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 17:07:56.778877 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777336    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 17:07:56.778877 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777339    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 17:07:56.778877 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777341    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 17:07:56.778877 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777344    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 17:07:56.778877 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777346    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 17:07:56.778877 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777349    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 17:07:56.778877 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777352    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 17:07:56.779396 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777355    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 17:07:56.779396 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777357    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 17:07:56.779396 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777360    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 17:07:56.779396 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777362    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 17:07:56.779396 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777364    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 17:07:56.779396 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777368    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 17:07:56.779396 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777373    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 17:07:56.779396 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777376    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 17:07:56.779396 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777379    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 17:07:56.779396 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777382    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 17:07:56.779396 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777385    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 17:07:56.779396 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777388    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 17:07:56.779396 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777391    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 17:07:56.779396 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777394    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 17:07:56.779396 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777396    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 17:07:56.779396 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777399    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 17:07:56.779396 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777402    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 17:07:56.779396 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777405    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 17:07:56.779396 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777408    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 17:07:56.779396 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777411    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 17:07:56.779896 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777414    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 17:07:56.779896 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777419    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 17:07:56.779896 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777423    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 17:07:56.779896 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777426    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 17:07:56.779896 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777429    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 17:07:56.779896 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777432    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 17:07:56.779896 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777435    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 17:07:56.779896 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777438    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 17:07:56.779896 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777440    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 17:07:56.779896 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777443    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 17:07:56.779896 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777446    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 17:07:56.779896 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777449    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 17:07:56.779896 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777451    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 17:07:56.779896 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777454    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 17:07:56.779896 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777456    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 17:07:56.779896 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777458    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 17:07:56.779896 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777461    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 17:07:56.779896 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777463    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 17:07:56.779896 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777466    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 17:07:56.780384 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777468    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 17:07:56.780384 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777472    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 17:07:56.780384 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777474    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 17:07:56.780384 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777476    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 17:07:56.780384 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777479    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 17:07:56.780384 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777482    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 17:07:56.780384 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777484    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 17:07:56.780384 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777487    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 17:07:56.780384 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777489    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 17:07:56.780384 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777492    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 17:07:56.780384 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777496    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 17:07:56.780384 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777499    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 17:07:56.780384 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777502    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 17:07:56.780384 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777504    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 17:07:56.780384 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777507    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 17:07:56.780384 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777510    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 17:07:56.780384 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777513    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 17:07:56.780384 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777515    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 17:07:56.780384 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777518    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 17:07:56.780384 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777521    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 17:07:56.780865 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777523    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 17:07:56.780865 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777526    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 17:07:56.780865 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777528    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 17:07:56.780865 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777531    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 17:07:56.780865 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777533    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 17:07:56.780865 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777535    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 17:07:56.780865 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777538    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 17:07:56.780865 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777541    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 17:07:56.780865 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777543    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 17:07:56.780865 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777546    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 17:07:56.780865 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777548    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 17:07:56.780865 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777551    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 17:07:56.780865 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777553    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 17:07:56.780865 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777556    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 17:07:56.780865 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777558    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 17:07:56.780865 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777561    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 17:07:56.780865 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777563    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 17:07:56.780865 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777566    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 17:07:56.780865 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777568    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 17:07:56.780865 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777976    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 17:07:56.781373 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777981    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 17:07:56.781373 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777984    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 17:07:56.781373 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777986    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 17:07:56.781373 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777989    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 17:07:56.781373 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777991    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 17:07:56.781373 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777994    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 17:07:56.781373 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.777997    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 17:07:56.781373 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778000    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 17:07:56.781373 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778003    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 17:07:56.781373 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778006    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 17:07:56.781373 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778009    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 17:07:56.781373 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778012    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 17:07:56.781373 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778015    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 17:07:56.781373 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778017    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 17:07:56.781373 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778020    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 17:07:56.781373 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778023    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 17:07:56.781373 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778025    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 17:07:56.781373 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778030    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 17:07:56.781373 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778033    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 17:07:56.781373 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778035    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 17:07:56.781867 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778038    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 17:07:56.781867 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778040    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 17:07:56.781867 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778043    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 17:07:56.781867 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778045    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 17:07:56.781867 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778048    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 17:07:56.781867 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778050    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 17:07:56.781867 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778052    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 17:07:56.781867 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778055    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 17:07:56.781867 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778059    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 17:07:56.781867 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778062    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 17:07:56.781867 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778065    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 17:07:56.781867 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778067    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 17:07:56.781867 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778070    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 17:07:56.781867 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778073    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 17:07:56.781867 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778076    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 17:07:56.781867 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778078    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 17:07:56.781867 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778081    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 17:07:56.781867 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778084    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 17:07:56.781867 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778087    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 17:07:56.781867 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778089    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 17:07:56.782367 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778092    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 17:07:56.782367 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778095    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 17:07:56.782367 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778098    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 17:07:56.782367 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778100    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 17:07:56.782367 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778103    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 17:07:56.782367 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778107    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 17:07:56.782367 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778110    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 17:07:56.782367 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778114    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 17:07:56.782367 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778116    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 17:07:56.782367 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778120    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 17:07:56.782367 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778123    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 17:07:56.782367 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778127    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 17:07:56.782367 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778130    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 17:07:56.782367 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778132    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 17:07:56.782367 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778135    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 17:07:56.782367 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778137    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 17:07:56.782367 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778140    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 17:07:56.782367 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778142    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 17:07:56.782367 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778145    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 17:07:56.782826 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778148    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 17:07:56.782826 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778150    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 17:07:56.782826 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778153    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 17:07:56.782826 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778155    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 17:07:56.782826 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778158    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 17:07:56.782826 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778161    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 17:07:56.782826 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778165    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 17:07:56.782826 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778169    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 17:07:56.782826 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778171    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 17:07:56.782826 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778174    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 17:07:56.782826 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778177    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 17:07:56.782826 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778179    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 17:07:56.782826 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778182    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 17:07:56.782826 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778184    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 17:07:56.782826 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778187    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 17:07:56.782826 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778190    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 17:07:56.782826 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778193    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 17:07:56.782826 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778195    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 17:07:56.782826 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778197    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 17:07:56.783318 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778200    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 17:07:56.783318 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778203    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 17:07:56.783318 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778205    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 17:07:56.783318 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778209    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 17:07:56.783318 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778212    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 17:07:56.783318 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778214    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 17:07:56.783318 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778216    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 17:07:56.783318 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778219    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 17:07:56.783318 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778221    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 17:07:56.783318 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778224    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 17:07:56.783318 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778226    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 17:07:56.783318 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778229    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 17:07:56.783318 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778231    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 17:07:56.783318 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778233    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 17:07:56.783318 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778236    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 17:07:56.783318 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778238    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 17:07:56.783318 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778241    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 17:07:56.783318 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778243    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 17:07:56.783318 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778246    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 17:07:56.783781 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778250    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 17:07:56.783781 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778321    2579 flags.go:64] FLAG: --address="0.0.0.0"
Jul 07 17:07:56.783781 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778332    2579 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 07 17:07:56.783781 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778338    2579 flags.go:64] FLAG: --anonymous-auth="true"
Jul 07 17:07:56.783781 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778344    2579 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 07 17:07:56.783781 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778348    2579 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 07 17:07:56.783781 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778352    2579 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 07 17:07:56.783781 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778358    2579 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 07 17:07:56.783781 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778363    2579 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 07 17:07:56.783781 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778367    2579 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 07 17:07:56.783781 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778370    2579 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 07 17:07:56.783781 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778373    2579 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 07 17:07:56.783781 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778377    2579 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 07 17:07:56.783781 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778380    2579 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 07 17:07:56.783781 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778383    2579 flags.go:64] FLAG: --cgroup-root=""
Jul 07 17:07:56.783781 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778385    2579 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 07 17:07:56.783781 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778390    2579 flags.go:64] FLAG: --client-ca-file=""
Jul 07 17:07:56.783781 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778393    2579 flags.go:64] FLAG: --cloud-provider="external"
Jul 07 17:07:56.783781 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778396    2579 flags.go:64] FLAG: --cluster-dns="[]"
Jul 07 17:07:56.783781 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778401    2579 flags.go:64] FLAG: --cluster-domain=""
Jul 07 17:07:56.783781 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778404    2579 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 07 17:07:56.783781 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778407    2579 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 07 17:07:56.783781 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778410    2579 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 07 17:07:56.783781 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778414    2579 flags.go:64] FLAG: --container-log-max-files="5"
Jul 07 17:07:56.784367 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778418    2579 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 07 17:07:56.784367 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778421    2579 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 07 17:07:56.784367 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778424    2579 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 07 17:07:56.784367 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778427    2579 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 07 17:07:56.784367 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778430    2579 flags.go:64] FLAG: --contention-profiling="false"
Jul 07 17:07:56.784367 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778433    2579 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 07 17:07:56.784367 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778436    2579 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 07 17:07:56.784367 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778439    2579 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 07 17:07:56.784367 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778444    2579 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 07 17:07:56.784367 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778448    2579 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 07 17:07:56.784367 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778451    2579 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 07 17:07:56.784367 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778453    2579 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 07 17:07:56.784367 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778456    2579 flags.go:64] FLAG: --enable-load-reader="false"
Jul 07 17:07:56.784367 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778459    2579 flags.go:64] FLAG: --enable-server="true"
Jul 07 17:07:56.784367 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778462    2579 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 07 17:07:56.784367 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778467    2579 flags.go:64] FLAG: --event-burst="100"
Jul 07 17:07:56.784367 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778470    2579 flags.go:64] FLAG: --event-qps="50"
Jul 07 17:07:56.784367 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778473    2579 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 07 17:07:56.784367 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778477    2579 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 07 17:07:56.784367 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778480    2579 flags.go:64] FLAG: --eviction-hard=""
Jul 07 17:07:56.784367 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778485    2579 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 07 17:07:56.784367 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778487    2579 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 07 17:07:56.784367 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778490    2579 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 07 17:07:56.784367 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778493    2579 flags.go:64] FLAG: --eviction-soft=""
Jul 07 17:07:56.784367 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778497    2579 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 07 17:07:56.784996 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778500    2579 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 07 17:07:56.784996 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778502    2579 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 07 17:07:56.784996 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778505    2579 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 07 17:07:56.784996 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778508    2579 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 07 17:07:56.784996 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778510    2579 flags.go:64] FLAG: --fail-swap-on="true"
Jul 07 17:07:56.784996 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778513    2579 flags.go:64] FLAG: --feature-gates=""
Jul 07 17:07:56.784996 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778517    2579 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 07 17:07:56.784996 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778520    2579 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 07 17:07:56.784996 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778523    2579 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 07 17:07:56.784996 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778526    2579 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 07 17:07:56.784996 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778529    2579 flags.go:64] FLAG: --healthz-port="10248"
Jul 07 17:07:56.784996 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778532    2579 flags.go:64] FLAG: --help="false"
Jul 07 17:07:56.784996 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778535    2579 flags.go:64] FLAG: --hostname-override="ip-10-0-142-103.ec2.internal"
Jul 07 17:07:56.784996 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778538    2579 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 07 17:07:56.784996 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778541    2579 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 07 17:07:56.784996 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778546    2579 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 07 17:07:56.784996 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778549    2579 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 07 17:07:56.784996 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778552    2579 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 07 17:07:56.784996 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778555    2579 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 07 17:07:56.784996 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778558    2579 flags.go:64] FLAG: --image-service-endpoint=""
Jul 07 17:07:56.784996 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778561    2579 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 07 17:07:56.784996 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778563    2579 flags.go:64] FLAG: --kube-api-burst="100"
Jul 07 17:07:56.784996 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778566    2579 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 07 17:07:56.785643 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778569    2579 flags.go:64] FLAG: --kube-api-qps="50"
Jul 07 17:07:56.785643 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778572    2579 flags.go:64] FLAG: --kube-reserved=""
Jul 07 17:07:56.785643 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778575    2579 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 07 17:07:56.785643 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778578    2579 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 07 17:07:56.785643 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778581    2579 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 07 17:07:56.785643 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778584    2579 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 07 17:07:56.785643 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778587    2579 flags.go:64] FLAG: --lock-file=""
Jul 07 17:07:56.785643 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778589    2579 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 07 17:07:56.785643 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778593    2579 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 07 17:07:56.785643 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778596    2579 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 07 17:07:56.785643 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778601    2579 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 07 17:07:56.785643 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778604    2579 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 07 17:07:56.785643 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778607    2579 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 07 17:07:56.785643 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778609    2579 flags.go:64] FLAG: --logging-format="text"
Jul 07 17:07:56.785643 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778612    2579 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 07 17:07:56.785643 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778616    2579 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 07 17:07:56.785643 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778619    2579 flags.go:64] FLAG: --manifest-url=""
Jul 07 17:07:56.785643 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778622    2579 flags.go:64] FLAG: --manifest-url-header=""
Jul 07 17:07:56.785643 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778626    2579 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 07 17:07:56.785643 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778629    2579 flags.go:64] FLAG: --max-open-files="1000000"
Jul 07 17:07:56.785643 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778634    2579 flags.go:64] FLAG: --max-pods="110"
Jul 07 17:07:56.785643 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778637    2579 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 07 17:07:56.785643 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778640    2579 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 07 17:07:56.785643 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778642    2579 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 07 17:07:56.785643 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778647    2579 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 07 17:07:56.786287 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778650    2579 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 07 17:07:56.786287 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778653    2579 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 07 17:07:56.786287 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778656    2579 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 07 17:07:56.786287 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778663    2579 flags.go:64] FLAG: --node-status-max-images="50"
Jul 07 17:07:56.786287 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778666    2579 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 07 17:07:56.786287 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778669    2579 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 07 17:07:56.786287 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778672    2579 flags.go:64] FLAG: --pod-cidr=""
Jul 07 17:07:56.786287 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778675    2579 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 07 17:07:56.786287 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778681    2579 flags.go:64] FLAG: --pod-manifest-path=""
Jul 07 17:07:56.786287 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778684    2579 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 07 17:07:56.786287 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778687    2579 flags.go:64] FLAG: --pods-per-core="0"
Jul 07 17:07:56.786287 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778691    2579 flags.go:64] FLAG: --port="10250"
Jul 07 17:07:56.786287 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778694    2579 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 07 17:07:56.786287 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778696    2579 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0311bf6d351c2d1ae"
Jul 07 17:07:56.786287 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778699    2579 flags.go:64] FLAG: --qos-reserved=""
Jul 07 17:07:56.786287 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778703    2579 flags.go:64] FLAG: --read-only-port="10255"
Jul 07 17:07:56.786287 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778706    2579 flags.go:64] FLAG: --register-node="true"
Jul 07 17:07:56.786287 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778709    2579 flags.go:64] FLAG: --register-with-taints=""
Jul 07 17:07:56.786287 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778712    2579 flags.go:64] FLAG: --registry-burst="10"
Jul 07 17:07:56.786287 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778715    2579 flags.go:64] FLAG: --registry-qps="5"
Jul 07 17:07:56.786287 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778718    2579 flags.go:64] FLAG: --reserved-cpus=""
Jul 07 17:07:56.786287 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778720    2579 flags.go:64] FLAG: --reserved-memory=""
Jul 07 17:07:56.786287 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778724    2579 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 07 17:07:56.786287 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778727    2579 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 07 17:07:56.786287 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778730    2579 flags.go:64] FLAG: --rotate-certificates="false"
Jul 07 17:07:56.786911 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778733    2579 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 07 17:07:56.786911 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778735    2579 flags.go:64] FLAG: --runonce="false"
Jul 07 17:07:56.786911 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778738    2579 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 07 17:07:56.786911 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778741    2579 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 07 17:07:56.786911 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778744    2579 flags.go:64] FLAG: --seccomp-default="false"
Jul 07 17:07:56.786911 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778747    2579 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 07 17:07:56.786911 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778750    2579 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 07 17:07:56.786911 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778755    2579 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 07 17:07:56.786911 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778757    2579 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 07 17:07:56.786911 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778760    2579 flags.go:64] FLAG: --storage-driver-password="root"
Jul 07 17:07:56.786911 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778763    2579 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 07 17:07:56.786911 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778766    2579 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 07 17:07:56.786911 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778769    2579 flags.go:64] FLAG: --storage-driver-user="root"
Jul 07 17:07:56.786911 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778772    2579 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 07 17:07:56.786911 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778775    2579 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 07 17:07:56.786911 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778778    2579 flags.go:64] FLAG: --system-cgroups=""
Jul 07 17:07:56.786911 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778780    2579 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 07 17:07:56.786911 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778785    2579 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 07 17:07:56.786911 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778788    2579 flags.go:64] FLAG: --tls-cert-file=""
Jul 07 17:07:56.786911 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778791    2579 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 07 17:07:56.786911 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778796    2579 flags.go:64] FLAG: --tls-min-version=""
Jul 07 17:07:56.786911 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778799    2579 flags.go:64] FLAG: --tls-private-key-file=""
Jul 07 17:07:56.786911 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778802    2579 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 07 17:07:56.786911 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778804    2579 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 07 17:07:56.786911 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778807    2579 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 07 17:07:56.787523 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778810    2579 flags.go:64] FLAG: --v="2"
Jul 07 17:07:56.787523 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778814    2579 flags.go:64] FLAG: --version="false"
Jul 07 17:07:56.787523 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778818    2579 flags.go:64] FLAG: --vmodule=""
Jul 07 17:07:56.787523 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778821    2579 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 07 17:07:56.787523 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.778824    2579 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 07 17:07:56.787523 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778954    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 17:07:56.787523 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778958    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 17:07:56.787523 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778961    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 17:07:56.787523 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778963    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 17:07:56.787523 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778966    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 17:07:56.787523 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778969    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 17:07:56.787523 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778971    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 17:07:56.787523 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778974    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 17:07:56.787523 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778977    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 17:07:56.787523 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778980    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 17:07:56.787523 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778982    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 17:07:56.787523 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778985    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 17:07:56.787523 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778987    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 17:07:56.787523 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778990    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 17:07:56.787523 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778992    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 17:07:56.787523 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778996    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 17:07:56.788061 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.778998    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 17:07:56.788061 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779001    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 17:07:56.788061 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779004    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 17:07:56.788061 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779007    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 17:07:56.788061 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779009    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 17:07:56.788061 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779011    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 17:07:56.788061 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779015    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 17:07:56.788061 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779019    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 17:07:56.788061 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779022    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 17:07:56.788061 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779025    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 17:07:56.788061 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779028    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 17:07:56.788061 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779031    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 17:07:56.788061 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779033    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 17:07:56.788061 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779036    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 17:07:56.788061 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779038    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 17:07:56.788061 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779041    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 17:07:56.788061 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779043    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 17:07:56.788061 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779046    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 17:07:56.788061 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779049    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 17:07:56.788539 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779053    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 17:07:56.788539 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779056    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 17:07:56.788539 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779060    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 17:07:56.788539 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779063    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 17:07:56.788539 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779065    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 17:07:56.788539 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779067    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 17:07:56.788539 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779070    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 17:07:56.788539 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779072    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 17:07:56.788539 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779075    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 17:07:56.788539 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779077    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 17:07:56.788539 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779079    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 17:07:56.788539 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779082    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 17:07:56.788539 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779086    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 17:07:56.788539 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779088    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 17:07:56.788539 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779091    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 17:07:56.788539 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779093    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 17:07:56.788539 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779095    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 17:07:56.788539 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779098    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 17:07:56.788539 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779100    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 17:07:56.788539 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779103    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 17:07:56.789089 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779109    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 17:07:56.789089 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779111    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 17:07:56.789089 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779114    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 17:07:56.789089 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779117    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 17:07:56.789089 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779119    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 17:07:56.789089 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779122    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 17:07:56.789089 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779124    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 17:07:56.789089 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779127    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 17:07:56.789089 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779129    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 17:07:56.789089 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779132    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 17:07:56.789089 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779134    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 17:07:56.789089 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779137    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 17:07:56.789089 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779139    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 17:07:56.789089 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779142    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 17:07:56.789089 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779144    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 17:07:56.789089 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779147    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 17:07:56.789089 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779149    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 17:07:56.789089 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779151    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 17:07:56.789089 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779154    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 17:07:56.789089 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779157    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 17:07:56.789585 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779159    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 17:07:56.789585 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779161    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 17:07:56.789585 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779164    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 17:07:56.789585 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779166    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 17:07:56.789585 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779171    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 17:07:56.789585 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779173    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 17:07:56.789585 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779176    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 17:07:56.789585 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779178    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 17:07:56.789585 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779181    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 17:07:56.789585 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779184    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 17:07:56.789585 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779186    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 17:07:56.789585 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779188    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 17:07:56.789585 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779192    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 17:07:56.789585 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779196    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 17:07:56.789585 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779199    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 17:07:56.789585 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779202    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 17:07:56.789585 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779205    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 17:07:56.789585 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779207    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 17:07:56.789585 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779210    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 17:07:56.790073 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779213    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 17:07:56.790073 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779216    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 17:07:56.790073 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779218    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 17:07:56.790073 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779221    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 17:07:56.790073 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.779224    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 17:07:56.790073 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.779847    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 17:07:56.790073 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.786836    2579 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 07 17:07:56.790073 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.786853    2579 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 07 17:07:56.790073 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.787596    2579 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 07 17:07:56.790073 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.787606    2579 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 07 17:07:56.790073 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787666    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 17:07:56.790073 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787673    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 17:07:56.790073 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787678    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 17:07:56.790073 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787681    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 17:07:56.790432 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787684    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 17:07:56.790432 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787687    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 17:07:56.790432 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787690    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 17:07:56.790432 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787693    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 17:07:56.790432 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787696    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 17:07:56.790432 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787699    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 17:07:56.790432 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787701    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 17:07:56.790432 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787704    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 17:07:56.790432 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787707    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 17:07:56.790432 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787710    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 17:07:56.790432 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787713    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 17:07:56.790432 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787715    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 17:07:56.790432 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787719    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 17:07:56.790432 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787722    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 17:07:56.790432 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787725    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 17:07:56.790432 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787727    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 17:07:56.790432 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787730    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 17:07:56.790432 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787732    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 17:07:56.790432 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787735    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 17:07:56.790432 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787738    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 17:07:56.790926 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787740    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 17:07:56.790926 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787742    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 17:07:56.790926 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787745    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 17:07:56.790926 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787747    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 17:07:56.790926 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787750    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 17:07:56.790926 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787753    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 17:07:56.790926 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787757    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 17:07:56.790926 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787759    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 17:07:56.790926 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787762    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 17:07:56.790926 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787765    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 17:07:56.790926 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787767    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 17:07:56.790926 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787769    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 17:07:56.790926 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787772    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 17:07:56.790926 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787774    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 17:07:56.790926 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787778    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 17:07:56.790926 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787782    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 17:07:56.790926 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787785    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 17:07:56.790926 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787787    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 17:07:56.790926 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787790    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 17:07:56.791398 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787793    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 17:07:56.791398 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787795    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 17:07:56.791398 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787798    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 17:07:56.791398 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787800    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 17:07:56.791398 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787804    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 17:07:56.791398 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787806    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 17:07:56.791398 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787826    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 17:07:56.791398 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787830    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 17:07:56.791398 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787834    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 17:07:56.791398 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787837    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 17:07:56.791398 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787839    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 17:07:56.791398 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787843    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 17:07:56.791398 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787845    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 17:07:56.791398 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787848    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 17:07:56.791398 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787851    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 17:07:56.791398 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787854    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 17:07:56.791398 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787857    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 17:07:56.791398 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787859    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 17:07:56.791398 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787863    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 17:07:56.791398 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787866    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 17:07:56.791883 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787869    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 17:07:56.791883 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787871    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 17:07:56.791883 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787873    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 17:07:56.791883 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787876    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 17:07:56.791883 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787879    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 17:07:56.791883 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787882    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 17:07:56.791883 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787899    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 17:07:56.791883 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787903    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 17:07:56.791883 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787905    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 17:07:56.791883 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787908    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 17:07:56.791883 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787912    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 17:07:56.791883 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787914    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 17:07:56.791883 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787917    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 17:07:56.791883 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787920    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 17:07:56.791883 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787922    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 17:07:56.791883 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787925    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 17:07:56.791883 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787928    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 17:07:56.791883 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787930    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 17:07:56.791883 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787933    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 17:07:56.792386 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787936    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 17:07:56.792386 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787939    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 17:07:56.792386 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787941    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 17:07:56.792386 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787945    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 17:07:56.792386 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787949    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 17:07:56.792386 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787951    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 17:07:56.792386 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787954    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 17:07:56.792386 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787956    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 17:07:56.792386 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787959    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 17:07:56.792386 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787961    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 17:07:56.792386 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787963    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 17:07:56.792386 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787968    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 17:07:56.792386 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787970    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 17:07:56.792386 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787973    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 17:07:56.792386 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787975    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 17:07:56.792386 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787977    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 17:07:56.792386 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.787980    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 17:07:56.792840 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.787985    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 17:07:56.792840 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788106    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 17:07:56.792840 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788111    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 17:07:56.792840 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788114    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 17:07:56.792840 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788117    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 17:07:56.792840 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788120    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 17:07:56.792840 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788122    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 17:07:56.792840 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788125    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 17:07:56.792840 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788128    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 17:07:56.792840 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788130    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 17:07:56.792840 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788133    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 17:07:56.792840 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788136    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 17:07:56.792840 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788138    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 17:07:56.792840 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788141    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 17:07:56.792840 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788143    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 17:07:56.793223 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788146    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 17:07:56.793223 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788149    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 17:07:56.793223 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788151    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 17:07:56.793223 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788154    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 17:07:56.793223 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788156    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 17:07:56.793223 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788159    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 17:07:56.793223 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788161    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 17:07:56.793223 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788163    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 17:07:56.793223 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788166    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 17:07:56.793223 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788168    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 17:07:56.793223 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788171    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 17:07:56.793223 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788173    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 17:07:56.793223 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788176    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 17:07:56.793223 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788178    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 17:07:56.793223 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788181    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 17:07:56.793223 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788183    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 17:07:56.793223 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788186    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 17:07:56.793223 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788188    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 17:07:56.793223 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788191    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 17:07:56.793685 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788193    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 17:07:56.793685 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788196    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 17:07:56.793685 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788198    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 17:07:56.793685 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788201    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 17:07:56.793685 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788203    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 17:07:56.793685 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788205    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 17:07:56.793685 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788208    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 17:07:56.793685 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788210    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 17:07:56.793685 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788213    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 17:07:56.793685 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788215    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 17:07:56.793685 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788218    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 17:07:56.793685 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788220    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 17:07:56.793685 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788222    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 17:07:56.793685 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788225    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 17:07:56.793685 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788227    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 17:07:56.793685 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788230    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 17:07:56.793685 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788233    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 17:07:56.793685 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788235    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 17:07:56.793685 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788238    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 17:07:56.793685 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788240    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 17:07:56.794314 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788242    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 17:07:56.794314 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788244    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 17:07:56.794314 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788247    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 17:07:56.794314 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788249    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 17:07:56.794314 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788253    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 17:07:56.794314 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788257    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 17:07:56.794314 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788259    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 17:07:56.794314 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788262    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 17:07:56.794314 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788266    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 17:07:56.794314 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788269    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 17:07:56.794314 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788271    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 17:07:56.794314 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788274    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 17:07:56.794314 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788277    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 17:07:56.794314 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788279    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 17:07:56.794314 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788281    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 17:07:56.794314 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788284    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 17:07:56.794314 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788287    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 17:07:56.794314 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788290    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 17:07:56.794314 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788292    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 17:07:56.794314 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788294    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 17:07:56.794790 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788297    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 17:07:56.794790 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788299    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 17:07:56.794790 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788302    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 17:07:56.794790 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788305    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 17:07:56.794790 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788307    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 17:07:56.794790 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788309    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 17:07:56.794790 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788312    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 17:07:56.794790 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788314    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 17:07:56.794790 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788317    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 17:07:56.794790 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788320    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 17:07:56.794790 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788324    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 17:07:56.794790 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788327    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 17:07:56.794790 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788330    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 17:07:56.794790 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788332    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 17:07:56.794790 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788335    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 17:07:56.794790 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788338    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 17:07:56.794790 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788340    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 17:07:56.794790 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788342    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 17:07:56.794790 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788345    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 17:07:56.795295 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788347    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 17:07:56.795295 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788350    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 17:07:56.795295 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788352    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 17:07:56.795295 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788355    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 17:07:56.795295 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788359    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 17:07:56.795295 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788362    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 17:07:56.795295 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:56.788365    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 17:07:56.795295 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.788370    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 17:07:56.795295 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.788493    2579 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 07 17:07:56.795295 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.791353    2579 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 07 17:07:56.795295 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.792737    2579 server.go:1019] "Starting client certificate rotation"
Jul 07 17:07:56.795295 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.792834    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 07 17:07:56.795295 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.792873    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 07 17:07:56.795629 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.795186    2579 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 07 17:07:56.803332 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.803305    2579 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 07 17:07:56.811055 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.811031    2579 log.go:25] "Validated CRI v1 runtime API"
Jul 07 17:07:56.813029 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.813012    2579 log.go:25] "Validated CRI v1 image API"
Jul 07 17:07:56.813793 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.813775    2579 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 07 17:07:56.816985 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.816961    2579 fs.go:135] Filesystem UUIDs: map[2c72e193-4def-42f2-a371-87a8fcff9c8e:/dev/nvme0n1p4 42ad4a9f-998e-4061-978b-dda7096c5f99:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Jul 07 17:07:56.817031 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.816986    2579 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 07 17:07:56.822981 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.822844    2579 manager.go:216] Machine: {Timestamp:2026-07-07 17:07:56.821462529 +0000 UTC m=+0.317541028 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098555 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec28bdbb2c48de5bf568b9c5611d1432 SystemUUID:ec28bdbb-2c48-de5b-f568-b9c5611d1432 BootID:7b9f3cc7-4f43-487e-a993-4a681e0acbec Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:39:f0:c0:bc:3f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:39:f0:c0:bc:3f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:2e:34:a2:9d:18:26 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 07 17:07:56.822981 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.822969    2579 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 07 17:07:56.823097 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.823053    2579 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 07 17:07:56.825652 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.825627    2579 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 07 17:07:56.825803 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.825654    2579 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-103.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 07 17:07:56.826265 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.826255    2579 topology_manager.go:138] "Creating topology manager with none policy"
Jul 07 17:07:56.826303 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.826267    2579 container_manager_linux.go:309] "Creating device plugin manager"
Jul 07 17:07:56.826303 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.826275    2579 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 07 17:07:56.826577 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.826560    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 07 17:07:56.826851 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.826840    2579 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 07 17:07:56.828248 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.828237    2579 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 07 17:07:56.830883 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.830871    2579 state_mem.go:36] "Initialized new in-memory state store"
Jul 07 17:07:56.831024 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.831015    2579 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 07 17:07:56.831054 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.831049    2579 kubelet.go:486] "Attempting to sync node with API server"
Jul 07 17:07:56.831084 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.831062    2579 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 07 17:07:56.831084 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.831076    2579 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 07 17:07:56.831134 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.831086    2579 kubelet.go:398] "Adding apiserver pod source"
Jul 07 17:07:56.831134 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.831100    2579 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 07 17:07:56.831593 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.831567    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 07 17:07:56.831691 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.831601    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 07 17:07:56.833745 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.833724    2579 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 07 17:07:56.835702 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.835684    2579 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 07 17:07:56.835795 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.835725    2579 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 07 17:07:56.836918 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.836902    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 07 17:07:56.836990 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.836926    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 07 17:07:56.836990 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.836935    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 07 17:07:56.836990 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.836944    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 07 17:07:56.836990 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.836956    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 07 17:07:56.836990 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.836964    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 07 17:07:56.836990 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.836976    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 07 17:07:56.836990 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.836984    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 07 17:07:56.837213 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.836999    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 07 17:07:56.837213 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.837009    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 07 17:07:56.837213 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.837021    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 07 17:07:56.837213 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.837030    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 07 17:07:56.837815 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.837801    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 07 17:07:56.837871 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.837819    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 07 17:07:56.841492 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.841473    2579 server.go:1268] "Started kubelet"
Jul 07 17:07:56.841592 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.841563    2579 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 07 17:07:56.841657 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.841605    2579 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 07 17:07:56.841713 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.841662    2579 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 07 17:07:56.845774 ip-10-0-142-103 systemd[1]: Started Kubernetes Kubelet.
Jul 07 17:07:56.845941 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.845814    2579 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 07 17:07:56.846158 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:07:56.846134    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-103.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 07 17:07:56.846576 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:07:56.846521    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 07 17:07:56.847291 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.847255    2579 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-103.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 07 17:07:56.848866 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.848847    2579 server.go:310] "Adding debug handlers to kubelet server"
Jul 07 17:07:56.851504 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:07:56.851482    2579 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 07 17:07:56.853144 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.853125    2579 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 07 17:07:56.853261 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.853143    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 07 17:07:56.853261 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:07:56.853210    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-103.ec2.internal\" not found"
Jul 07 17:07:56.853743 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.853723    2579 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 07 17:07:56.853873 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.853766    2579 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 07 17:07:56.853992 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.853976    2579 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 07 17:07:56.854072 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:07:56.852913    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-103.ec2.internal.18c010f054e851f8  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-103.ec2.internal,UID:ip-10-0-142-103.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-103.ec2.internal,},FirstTimestamp:2026-07-07 17:07:56.841423352 +0000 UTC m=+0.337501910,LastTimestamp:2026-07-07 17:07:56.841423352 +0000 UTC m=+0.337501910,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-103.ec2.internal,}"
Jul 07 17:07:56.854200 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.854117    2579 factory.go:55] Registering systemd factory
Jul 07 17:07:56.854270 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.854264    2579 factory.go:223] Registration of the systemd container factory successfully
Jul 07 17:07:56.854330 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.854141    2579 reconstruct.go:97] "Volume reconstruction finished"
Jul 07 17:07:56.854330 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.854304    2579 reconciler.go:29] "Reconciler: start to sync state"
Jul 07 17:07:56.854460 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.854443    2579 factory.go:153] Registering CRI-O factory
Jul 07 17:07:56.854460 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.854460    2579 factory.go:223] Registration of the crio container factory successfully
Jul 07 17:07:56.854577 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.854512    2579 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 07 17:07:56.854577 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.854530    2579 factory.go:103] Registering Raw factory
Jul 07 17:07:56.854577 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.854541    2579 manager.go:1196] Started watching for new ooms in manager
Jul 07 17:07:56.855109 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.855088    2579 manager.go:349] Starting recovery of all containers
Jul 07 17:07:56.861990 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:07:56.861674    2579 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-103.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 07 17:07:56.861990 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:07:56.861995    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 07 17:07:56.865422 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.865404    2579 manager.go:354] Recovery completed
Jul 07 17:07:56.867215 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.867195    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8lrmd"
Jul 07 17:07:56.869831 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.869818    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 17:07:56.872331 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.872317    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-103.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 17:07:56.872394 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.872345    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-103.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 17:07:56.872394 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.872356    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-103.ec2.internal" event="NodeHasSufficientPID"
Jul 07 17:07:56.872952 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.872935    2579 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 07 17:07:56.872952 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.872950    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 17:07:56.873092 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.872950    2579 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 07 17:07:56.873092 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.872988    2579 state_mem.go:36] "Initialized new in-memory state store"
Jul 07 17:07:56.874059 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.874037    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-103.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 17:07:56.874120 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.874072    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-103.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 17:07:56.874120 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.874087    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-103.ec2.internal" event="NodeHasSufficientPID"
Jul 07 17:07:56.874230 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.874220    2579 policy_none.go:49] "None policy: Start"
Jul 07 17:07:56.874283 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.874235    2579 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 07 17:07:56.874283 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.874246    2579 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 07 17:07:56.875053 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.875042    2579 policy_none.go:47] "Start"
Jul 07 17:07:56.875627 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:07:56.875226    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-103.ec2.internal.18c010f056bff330  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-103.ec2.internal,UID:ip-10-0-142-103.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-103.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-103.ec2.internal,},FirstTimestamp:2026-07-07 17:07:56.87233208 +0000 UTC m=+0.368410583,LastTimestamp:2026-07-07 17:07:56.87233208 +0000 UTC m=+0.368410583,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-103.ec2.internal,}"
Jul 07 17:07:56.876731 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.876704    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8lrmd"
Jul 07 17:07:56.918549 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.918526    2579 manager.go:337] "Starting Device Plugin manager"
Jul 07 17:07:56.918681 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:07:56.918572    2579 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 07 17:07:56.918681 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.918585    2579 server.go:87] "Starting device plugin registration server"
Jul 07 17:07:56.918921 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.918907    2579 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 07 17:07:56.918987 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.918924    2579 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 07 17:07:56.919041 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.919005    2579 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 07 17:07:56.919097 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.919084    2579 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 07 17:07:56.919097 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:56.919092    2579 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 07 17:07:56.919654 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:07:56.919631    2579 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 07 17:07:56.919761 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:07:56.919675    2579 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-103.ec2.internal\" not found"
Jul 07 17:07:57.019065 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:57.019024    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 17:07:57.020266 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:57.019686    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 07 17:07:57.021172 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:57.021149    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 07 17:07:57.021271 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:57.021200    2579 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 07 17:07:57.021271 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:57.021226    2579 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 07 17:07:57.021271 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:57.021244    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-103.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 17:07:57.021415 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:57.021276    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-103.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 17:07:57.021415 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:07:57.021264    2579 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 07 17:07:57.021415 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:57.021293    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-103.ec2.internal" event="NodeHasSufficientPID"
Jul 07 17:07:57.021415 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:57.021330    2579 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-103.ec2.internal"
Jul 07 17:07:57.023804 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:57.023749    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 17:07:57.030999 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:57.030879    2579 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-103.ec2.internal"
Jul 07 17:07:57.031170 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:07:57.031009    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-103.ec2.internal\": node \"ip-10-0-142-103.ec2.internal\" not found"
Jul 07 17:07:57.080409 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:07:57.080376    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-103.ec2.internal\" not found"
Jul 07 17:07:57.121423 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:57.121389    2579 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-103.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-103.ec2.internal"]
Jul 07 17:07:57.121593 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:57.121488    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 17:07:57.123215 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:57.123199    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-103.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 17:07:57.123299 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:57.123229    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-103.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 17:07:57.123299 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:57.123240    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-103.ec2.internal" event="NodeHasSufficientPID"
Jul 07 17:07:57.124535 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:57.124522    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 17:07:57.124650 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:57.124637    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-103.ec2.internal"
Jul 07 17:07:57.124706 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:57.124668    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 17:07:57.125419 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:57.125401    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-103.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 17:07:57.125510 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:57.125437    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-103.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 17:07:57.125510 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:57.125450    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-103.ec2.internal" event="NodeHasSufficientPID"
Jul 07 17:07:57.126032 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:57.126020    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-103.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 17:07:57.126086 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:57.126045    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-103.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 17:07:57.126086 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:57.126055    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-103.ec2.internal" event="NodeHasSufficientPID"
Jul 07 17:07:57.126721 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:57.126705    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-103.ec2.internal"
Jul 07 17:07:57.126798 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:57.126738    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 17:07:57.127523 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:57.127509    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-103.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 17:07:57.127592 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:57.127539    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-103.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 17:07:57.127592 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:57.127552    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-103.ec2.internal" event="NodeHasSufficientPID"
Jul 07 17:07:57.156183 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:57.156150    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c3ab2f1cccd249a4a2fa498a1527f9d2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-103.ec2.internal\" (UID: \"c3ab2f1cccd249a4a2fa498a1527f9d2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-103.ec2.internal"
Jul 07 17:07:57.156183 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:57.156187    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c3ab2f1cccd249a4a2fa498a1527f9d2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-103.ec2.internal\" (UID: \"c3ab2f1cccd249a4a2fa498a1527f9d2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-103.ec2.internal"
Jul 07 17:07:57.156387 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:57.156211    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3df4ea321bf474671e41e2422091c862-config\") pod \"kube-apiserver-proxy-ip-10-0-142-103.ec2.internal\" (UID: \"3df4ea321bf474671e41e2422091c862\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-103.ec2.internal"
Jul 07 17:07:57.160735 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:07:57.160717    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-103.ec2.internal\" not found" node="ip-10-0-142-103.ec2.internal"
Jul 07 17:07:57.165055 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:07:57.165035    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-103.ec2.internal\" not found" node="ip-10-0-142-103.ec2.internal"
Jul 07 17:07:57.181110 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:07:57.181079    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-103.ec2.internal\" not found"
Jul 07 17:07:57.256685 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:57.256646    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c3ab2f1cccd249a4a2fa498a1527f9d2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-103.ec2.internal\" (UID: \"c3ab2f1cccd249a4a2fa498a1527f9d2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-103.ec2.internal"
Jul 07 17:07:57.256844 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:57.256683    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c3ab2f1cccd249a4a2fa498a1527f9d2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-103.ec2.internal\" (UID: \"c3ab2f1cccd249a4a2fa498a1527f9d2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-103.ec2.internal"
Jul 07 17:07:57.256844 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:57.256721    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3df4ea321bf474671e41e2422091c862-config\") pod \"kube-apiserver-proxy-ip-10-0-142-103.ec2.internal\" (UID: \"3df4ea321bf474671e41e2422091c862\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-103.ec2.internal"
Jul 07 17:07:57.256844 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:57.256759    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c3ab2f1cccd249a4a2fa498a1527f9d2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-103.ec2.internal\" (UID: \"c3ab2f1cccd249a4a2fa498a1527f9d2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-103.ec2.internal"
Jul 07 17:07:57.256844 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:57.256787    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c3ab2f1cccd249a4a2fa498a1527f9d2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-103.ec2.internal\" (UID: \"c3ab2f1cccd249a4a2fa498a1527f9d2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-103.ec2.internal"
Jul 07 17:07:57.256844 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:57.256765    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3df4ea321bf474671e41e2422091c862-config\") pod \"kube-apiserver-proxy-ip-10-0-142-103.ec2.internal\" (UID: \"3df4ea321bf474671e41e2422091c862\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-103.ec2.internal"
Jul 07 17:07:57.281955 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:07:57.281862    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-103.ec2.internal\" not found"
Jul 07 17:07:57.382799 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:07:57.382761    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-103.ec2.internal\" not found"
Jul 07 17:07:57.463265 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:57.463222    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-103.ec2.internal"
Jul 07 17:07:57.468031 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:57.468016    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-103.ec2.internal"
Jul 07 17:07:57.483835 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:07:57.483808    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-103.ec2.internal\" not found"
Jul 07 17:07:57.584347 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:07:57.584260    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-103.ec2.internal\" not found"
Jul 07 17:07:57.684911 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:07:57.684872    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-103.ec2.internal\" not found"
Jul 07 17:07:57.786842 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:07:57.786798    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-103.ec2.internal\" not found"
Jul 07 17:07:57.793068 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:57.793034    2579 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 07 17:07:57.793240 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:57.793196    2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 07 17:07:57.880567 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:57.880486    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 17:02:57 +0000 UTC" deadline="2027-12-06 19:56:02.188771095 +0000 UTC"
Jul 07 17:07:57.880567 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:57.880524    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12410h48m4.308250954s"
Jul 07 17:07:57.887767 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:07:57.887743    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-103.ec2.internal\" not found"
Jul 07 17:07:57.988717 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:07:57.988643    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-103.ec2.internal\" not found"
Jul 07 17:07:58.044248 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:58.044206    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3df4ea321bf474671e41e2422091c862.slice/crio-58de8a8e97f0534db9608d34a8e7714172c9ab72e902c6752c0e9f4bd2668c3f WatchSource:0}: Error finding container 58de8a8e97f0534db9608d34a8e7714172c9ab72e902c6752c0e9f4bd2668c3f: Status 404 returned error can't find the container with id 58de8a8e97f0534db9608d34a8e7714172c9ab72e902c6752c0e9f4bd2668c3f
Jul 07 17:07:58.044699 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:07:58.044678    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3ab2f1cccd249a4a2fa498a1527f9d2.slice/crio-9bfb71c96bcdb6918dae7bf607606cd2e9444a215594f418404dc60514780466 WatchSource:0}: Error finding container 9bfb71c96bcdb6918dae7bf607606cd2e9444a215594f418404dc60514780466: Status 404 returned error can't find the container with id 9bfb71c96bcdb6918dae7bf607606cd2e9444a215594f418404dc60514780466
Jul 07 17:07:58.049390 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.049368    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 17:07:58.068606 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.068574    2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 17:07:58.089201 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:07:58.089160    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-103.ec2.internal\" not found"
Jul 07 17:07:58.174189 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.174113    2579 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 17:07:58.189600 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:07:58.189571    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-103.ec2.internal\" not found"
Jul 07 17:07:58.290625 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:07:58.290582    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-103.ec2.internal\" not found"
Jul 07 17:07:58.392486 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:07:58.392449    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-103.ec2.internal\" not found"
Jul 07 17:07:58.420512 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.420484    2579 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 17:07:58.453970 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.453866    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-103.ec2.internal"
Jul 07 17:07:58.471312 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.471273    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 07 17:07:58.472428 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.472407    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-103.ec2.internal"
Jul 07 17:07:58.488916 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.488880    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 07 17:07:58.832522 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.832181    2579 apiserver.go:52] "Watching apiserver"
Jul 07 17:07:58.847032 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.846132    2579 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 07 17:07:58.847406 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.847299    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-j6qcn","kube-system/global-pull-secret-syncer-wqh28","kube-system/kube-apiserver-proxy-ip-10-0-142-103.ec2.internal","openshift-dns/node-resolver-66zv2","openshift-image-registry/node-ca-87n7p","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-103.ec2.internal","openshift-ovn-kubernetes/ovnkube-node-vhsvz","kube-system/konnectivity-agent-ktm2t","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vld2l","openshift-cluster-node-tuning-operator/tuned-2mpbm","openshift-multus/multus-additional-cni-plugins-4h5b6","openshift-multus/multus-ls9b9","openshift-multus/network-metrics-daemon-24rwk","openshift-network-diagnostics/network-check-target-fm4tg"]
Jul 07 17:07:58.850720 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.850692    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.853218 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.853197    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-24rwk"
Jul 07 17:07:58.853343 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:07:58.853277    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-24rwk" podUID="c398fff8-ec63-4063-b3a4-92bf0aa5df9c"
Jul 07 17:07:58.853412 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.853394    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 07 17:07:58.853732 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.853705    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 07 17:07:58.853866 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.853779    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 07 17:07:58.854044 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.854027    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 07 17:07:58.854672 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.854658    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 07 17:07:58.854778 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.854726    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-46lhr\""
Jul 07 17:07:58.854778 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.854755    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 07 17:07:58.854997 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.854954    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 07 17:07:58.856624 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.856487    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fm4tg"
Jul 07 17:07:58.856624 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:07:58.856583    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fm4tg" podUID="d6844e3d-9b85-4d30-86db-35466ed9a36e"
Jul 07 17:07:58.859426 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.859408    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-66zv2"
Jul 07 17:07:58.862750 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.862313    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-87n7p"
Jul 07 17:07:58.863084 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.863056    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-wtm4c\""
Jul 07 17:07:58.863198 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.863110    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 07 17:07:58.864005 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.863980    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 07 17:07:58.865246 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.865219    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ktm2t"
Jul 07 17:07:58.867400 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.867371    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4a99fb8d-bf83-4929-8838-c1649ba58cfb-host-slash\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.867512 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.867414    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4a99fb8d-bf83-4929-8838-c1649ba58cfb-host-run-netns\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.867577 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.867463    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a99fb8d-bf83-4929-8838-c1649ba58cfb-run-openvswitch\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.867636 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.867581    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4a99fb8d-bf83-4929-8838-c1649ba58cfb-run-ovn\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.867636 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.867611    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4a99fb8d-bf83-4929-8838-c1649ba58cfb-host-cni-bin\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.867744 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.867636    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4a99fb8d-bf83-4929-8838-c1649ba58cfb-host-cni-netd\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.867744 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.867702    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4a99fb8d-bf83-4929-8838-c1649ba58cfb-run-systemd\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.867744 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.867732    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a99fb8d-bf83-4929-8838-c1649ba58cfb-var-lib-openvswitch\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.867894 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.867756    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4a99fb8d-bf83-4929-8838-c1649ba58cfb-log-socket\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.867894 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.867778    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a99fb8d-bf83-4929-8838-c1649ba58cfb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.867894 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.867795    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4a99fb8d-bf83-4929-8838-c1649ba58cfb-ovnkube-config\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.867894 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.867845    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4a99fb8d-bf83-4929-8838-c1649ba58cfb-ovnkube-script-lib\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.867894 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.867875    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r54vz\" (UniqueName: \"kubernetes.io/projected/4a99fb8d-bf83-4929-8838-c1649ba58cfb-kube-api-access-r54vz\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.868184 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.867950    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbv8k\" (UniqueName: \"kubernetes.io/projected/d6844e3d-9b85-4d30-86db-35466ed9a36e-kube-api-access-jbv8k\") pod \"network-check-target-fm4tg\" (UID: \"d6844e3d-9b85-4d30-86db-35466ed9a36e\") " pod="openshift-network-diagnostics/network-check-target-fm4tg"
Jul 07 17:07:58.868184 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.867990    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4a99fb8d-bf83-4929-8838-c1649ba58cfb-node-log\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.868184 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.868016    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4a99fb8d-bf83-4929-8838-c1649ba58cfb-ovn-node-metrics-cert\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.868184 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.868055    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c398fff8-ec63-4063-b3a4-92bf0aa5df9c-metrics-certs\") pod \"network-metrics-daemon-24rwk\" (UID: \"c398fff8-ec63-4063-b3a4-92bf0aa5df9c\") " pod="openshift-multus/network-metrics-daemon-24rwk"
Jul 07 17:07:58.868184 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.868090    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m56zp\" (UniqueName: \"kubernetes.io/projected/c398fff8-ec63-4063-b3a4-92bf0aa5df9c-kube-api-access-m56zp\") pod \"network-metrics-daemon-24rwk\" (UID: \"c398fff8-ec63-4063-b3a4-92bf0aa5df9c\") " pod="openshift-multus/network-metrics-daemon-24rwk"
Jul 07 17:07:58.868184 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.868118    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a99fb8d-bf83-4929-8838-c1649ba58cfb-host-kubelet\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.868184 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.868142    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4a99fb8d-bf83-4929-8838-c1649ba58cfb-systemd-units\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.868184 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.868165    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a99fb8d-bf83-4929-8838-c1649ba58cfb-etc-openvswitch\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.868600 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.868204    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a99fb8d-bf83-4929-8838-c1649ba58cfb-host-run-ovn-kubernetes\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.868600 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.868250    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4a99fb8d-bf83-4929-8838-c1649ba58cfb-env-overrides\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.868600 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.868499    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vld2l"
Jul 07 17:07:58.871046 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.871024    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-j6qcn"
Jul 07 17:07:58.873987 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.873864    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqh28"
Jul 07 17:07:58.873987 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:07:58.873956    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqh28" podUID="5d706921-ddd7-4b39-840d-a03cef437d64"
Jul 07 17:07:58.874893 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.874857    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 07 17:07:58.875047 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.874856    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 07 17:07:58.875047 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.875012    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 07 17:07:58.875047 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.874857    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 07 17:07:58.875257 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.875119    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 07 17:07:58.875330 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.875313    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 07 17:07:58.875418 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.875398    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 07 17:07:58.876363 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.875928    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 07 17:07:58.876363 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.875895    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 07 17:07:58.876363 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.876056    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-l2799\""
Jul 07 17:07:58.876363 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.875926    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-8fdrb\""
Jul 07 17:07:58.876363 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.876266    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 07 17:07:58.876652 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.876374    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 07 17:07:58.876697 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.876669    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-xb2rc\""
Jul 07 17:07:58.876860 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.876845    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-fmd2k\""
Jul 07 17:07:58.878432 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.878415    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2mpbm"
Jul 07 17:07:58.879888 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.879865    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 17:07:58.882786 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.881604    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4h5b6"
Jul 07 17:07:58.884266 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.883991    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ls9b9"
Jul 07 17:07:58.893862 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.893840    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 07 17:07:58.894656 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.894640    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 07 17:07:58.895343 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.895327    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 07 17:07:58.897270 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.895547    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-dm86w\""
Jul 07 17:07:58.897270 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.895770    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 07 17:07:58.897270 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.895948    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 07 17:07:58.897270 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.896172    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 07 17:07:58.897270 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.896352    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 07 17:07:58.897270 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.896609    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 07 17:07:58.897270 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.896819    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-bwqpt\""
Jul 07 17:07:58.901361 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.898048    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-k4snr\""
Jul 07 17:07:58.901361 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.898262    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 07 17:07:58.951389 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.951355    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-7p4s7"
Jul 07 17:07:58.955329 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.955301    2579 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 07 17:07:58.963668 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.963644    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-7p4s7"
Jul 07 17:07:58.969114 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.969085    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a99fb8d-bf83-4929-8838-c1649ba58cfb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.969239 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.969120    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a99fb8d-bf83-4929-8838-c1649ba58cfb-var-lib-openvswitch\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.969239 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.969143    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4a99fb8d-bf83-4929-8838-c1649ba58cfb-ovnkube-config\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.969239 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.969170    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4a99fb8d-bf83-4929-8838-c1649ba58cfb-ovnkube-script-lib\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.969239 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.969196    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r54vz\" (UniqueName: \"kubernetes.io/projected/4a99fb8d-bf83-4929-8838-c1649ba58cfb-kube-api-access-r54vz\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.969239 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.969211    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a99fb8d-bf83-4929-8838-c1649ba58cfb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.969239 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.969225    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a4018c2b-fdcb-454a-b094-c2ed4e08e8ef-run\") pod \"tuned-2mpbm\" (UID: \"a4018c2b-fdcb-454a-b094-c2ed4e08e8ef\") " pod="openshift-cluster-node-tuning-operator/tuned-2mpbm"
Jul 07 17:07:58.969542 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.969217    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a99fb8d-bf83-4929-8838-c1649ba58cfb-var-lib-openvswitch\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.969542 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.969251    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5e44079c-1f38-4216-bb3b-9bee9ce4338d-host-run-netns\") pod \"multus-ls9b9\" (UID: \"5e44079c-1f38-4216-bb3b-9bee9ce4338d\") " pod="openshift-multus/multus-ls9b9"
Jul 07 17:07:58.969646 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.969631    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5e44079c-1f38-4216-bb3b-9bee9ce4338d-host-var-lib-cni-multus\") pod \"multus-ls9b9\" (UID: \"5e44079c-1f38-4216-bb3b-9bee9ce4338d\") " pod="openshift-multus/multus-ls9b9"
Jul 07 17:07:58.969701 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.969656    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5e44079c-1f38-4216-bb3b-9bee9ce4338d-etc-kubernetes\") pod \"multus-ls9b9\" (UID: \"5e44079c-1f38-4216-bb3b-9bee9ce4338d\") " pod="openshift-multus/multus-ls9b9"
Jul 07 17:07:58.969758 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.969717    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sf9t2\" (UniqueName: \"kubernetes.io/projected/187a8f3e-75c8-461a-a0f0-f3e3723d3a34-kube-api-access-sf9t2\") pod \"node-ca-87n7p\" (UID: \"187a8f3e-75c8-461a-a0f0-f3e3723d3a34\") " pod="openshift-image-registry/node-ca-87n7p"
Jul 07 17:07:58.969758 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.969722    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4a99fb8d-bf83-4929-8838-c1649ba58cfb-ovnkube-config\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.969758 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.969737    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a4018c2b-fdcb-454a-b094-c2ed4e08e8ef-etc-modprobe-d\") pod \"tuned-2mpbm\" (UID: \"a4018c2b-fdcb-454a-b094-c2ed4e08e8ef\") " pod="openshift-cluster-node-tuning-operator/tuned-2mpbm"
Jul 07 17:07:58.969911 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.969778    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a4018c2b-fdcb-454a-b094-c2ed4e08e8ef-host\") pod \"tuned-2mpbm\" (UID: \"a4018c2b-fdcb-454a-b094-c2ed4e08e8ef\") " pod="openshift-cluster-node-tuning-operator/tuned-2mpbm"
Jul 07 17:07:58.969911 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.969803    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ecf1ac63-1ea5-4590-8d60-b199e5eff959-tmp-dir\") pod \"node-resolver-66zv2\" (UID: \"ecf1ac63-1ea5-4590-8d60-b199e5eff959\") " pod="openshift-dns/node-resolver-66zv2"
Jul 07 17:07:58.969911 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.969822    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7cnj\" (UniqueName: \"kubernetes.io/projected/ecf1ac63-1ea5-4590-8d60-b199e5eff959-kube-api-access-x7cnj\") pod \"node-resolver-66zv2\" (UID: \"ecf1ac63-1ea5-4590-8d60-b199e5eff959\") " pod="openshift-dns/node-resolver-66zv2"
Jul 07 17:07:58.969911 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.969846    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1c55e58a-1b4a-4643-923c-c235327231ed-system-cni-dir\") pod \"multus-additional-cni-plugins-4h5b6\" (UID: \"1c55e58a-1b4a-4643-923c-c235327231ed\") " pod="openshift-multus/multus-additional-cni-plugins-4h5b6"
Jul 07 17:07:58.969911 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.969862    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4a99fb8d-bf83-4929-8838-c1649ba58cfb-ovnkube-script-lib\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.970115 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.969885    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1c55e58a-1b4a-4643-923c-c235327231ed-cni-binary-copy\") pod \"multus-additional-cni-plugins-4h5b6\" (UID: \"1c55e58a-1b4a-4643-923c-c235327231ed\") " pod="openshift-multus/multus-additional-cni-plugins-4h5b6"
Jul 07 17:07:58.970115 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.969940    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4a99fb8d-bf83-4929-8838-c1649ba58cfb-systemd-units\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.970115 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.969957    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a99fb8d-bf83-4929-8838-c1649ba58cfb-etc-openvswitch\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.970115 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.969976    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a4018c2b-fdcb-454a-b094-c2ed4e08e8ef-tmp\") pod \"tuned-2mpbm\" (UID: \"a4018c2b-fdcb-454a-b094-c2ed4e08e8ef\") " pod="openshift-cluster-node-tuning-operator/tuned-2mpbm"
Jul 07 17:07:58.970115 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.969991    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5e44079c-1f38-4216-bb3b-9bee9ce4338d-cnibin\") pod \"multus-ls9b9\" (UID: \"5e44079c-1f38-4216-bb3b-9bee9ce4338d\") " pod="openshift-multus/multus-ls9b9"
Jul 07 17:07:58.970115 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.970018    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5e44079c-1f38-4216-bb3b-9bee9ce4338d-hostroot\") pod \"multus-ls9b9\" (UID: \"5e44079c-1f38-4216-bb3b-9bee9ce4338d\") " pod="openshift-multus/multus-ls9b9"
Jul 07 17:07:58.970115 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.970045    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a99fb8d-bf83-4929-8838-c1649ba58cfb-etc-openvswitch\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.970115 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.970062    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4a99fb8d-bf83-4929-8838-c1649ba58cfb-systemd-units\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.970115 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.970047    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5e44079c-1f38-4216-bb3b-9bee9ce4338d-multus-daemon-config\") pod \"multus-ls9b9\" (UID: \"5e44079c-1f38-4216-bb3b-9bee9ce4338d\") " pod="openshift-multus/multus-ls9b9"
Jul 07 17:07:58.970115 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.970102    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/49ea513d-8038-4a64-923b-5a32649fa089-host-slash\") pod \"iptables-alerter-j6qcn\" (UID: \"49ea513d-8038-4a64-923b-5a32649fa089\") " pod="openshift-network-operator/iptables-alerter-j6qcn"
Jul 07 17:07:58.970503 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.970127    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1c55e58a-1b4a-4643-923c-c235327231ed-os-release\") pod \"multus-additional-cni-plugins-4h5b6\" (UID: \"1c55e58a-1b4a-4643-923c-c235327231ed\") " pod="openshift-multus/multus-additional-cni-plugins-4h5b6"
Jul 07 17:07:58.970503 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.970156    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/66c074da-39ba-4e2e-ada6-9f71c051f3ec-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vld2l\" (UID: \"66c074da-39ba-4e2e-ada6-9f71c051f3ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vld2l"
Jul 07 17:07:58.970503 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.970191    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5d706921-ddd7-4b39-840d-a03cef437d64-dbus\") pod \"global-pull-secret-syncer-wqh28\" (UID: \"5d706921-ddd7-4b39-840d-a03cef437d64\") " pod="kube-system/global-pull-secret-syncer-wqh28"
Jul 07 17:07:58.970503 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.970218    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4a99fb8d-bf83-4929-8838-c1649ba58cfb-run-systemd\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.970503 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.970243    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a4018c2b-fdcb-454a-b094-c2ed4e08e8ef-etc-kubernetes\") pod \"tuned-2mpbm\" (UID: \"a4018c2b-fdcb-454a-b094-c2ed4e08e8ef\") " pod="openshift-cluster-node-tuning-operator/tuned-2mpbm"
Jul 07 17:07:58.970503 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.970266    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e44079c-1f38-4216-bb3b-9bee9ce4338d-multus-cni-dir\") pod \"multus-ls9b9\" (UID: \"5e44079c-1f38-4216-bb3b-9bee9ce4338d\") " pod="openshift-multus/multus-ls9b9"
Jul 07 17:07:58.970503 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.970292    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m56zp\" (UniqueName: \"kubernetes.io/projected/c398fff8-ec63-4063-b3a4-92bf0aa5df9c-kube-api-access-m56zp\") pod \"network-metrics-daemon-24rwk\" (UID: \"c398fff8-ec63-4063-b3a4-92bf0aa5df9c\") " pod="openshift-multus/network-metrics-daemon-24rwk"
Jul 07 17:07:58.970503 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.970332    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/187a8f3e-75c8-461a-a0f0-f3e3723d3a34-serviceca\") pod \"node-ca-87n7p\" (UID: \"187a8f3e-75c8-461a-a0f0-f3e3723d3a34\") " pod="openshift-image-registry/node-ca-87n7p"
Jul 07 17:07:58.970503 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.970345    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4a99fb8d-bf83-4929-8838-c1649ba58cfb-run-systemd\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.970503 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.970360    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a4018c2b-fdcb-454a-b094-c2ed4e08e8ef-etc-systemd\") pod \"tuned-2mpbm\" (UID: \"a4018c2b-fdcb-454a-b094-c2ed4e08e8ef\") " pod="openshift-cluster-node-tuning-operator/tuned-2mpbm"
Jul 07 17:07:58.970503 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.970388    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6fd6\" (UniqueName: \"kubernetes.io/projected/49ea513d-8038-4a64-923b-5a32649fa089-kube-api-access-g6fd6\") pod \"iptables-alerter-j6qcn\" (UID: \"49ea513d-8038-4a64-923b-5a32649fa089\") " pod="openshift-network-operator/iptables-alerter-j6qcn"
Jul 07 17:07:58.970503 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.970419    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1c55e58a-1b4a-4643-923c-c235327231ed-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4h5b6\" (UID: \"1c55e58a-1b4a-4643-923c-c235327231ed\") " pod="openshift-multus/multus-additional-cni-plugins-4h5b6"
Jul 07 17:07:58.970503 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.970445    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a4018c2b-fdcb-454a-b094-c2ed4e08e8ef-etc-sysconfig\") pod \"tuned-2mpbm\" (UID: \"a4018c2b-fdcb-454a-b094-c2ed4e08e8ef\") " pod="openshift-cluster-node-tuning-operator/tuned-2mpbm"
Jul 07 17:07:58.970503 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.970477    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a4018c2b-fdcb-454a-b094-c2ed4e08e8ef-etc-sysctl-d\") pod \"tuned-2mpbm\" (UID: \"a4018c2b-fdcb-454a-b094-c2ed4e08e8ef\") " pod="openshift-cluster-node-tuning-operator/tuned-2mpbm"
Jul 07 17:07:58.970503 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.970504    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5e44079c-1f38-4216-bb3b-9bee9ce4338d-host-run-multus-certs\") pod \"multus-ls9b9\" (UID: \"5e44079c-1f38-4216-bb3b-9bee9ce4338d\") " pod="openshift-multus/multus-ls9b9"
Jul 07 17:07:58.971207 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.970533    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4a99fb8d-bf83-4929-8838-c1649ba58cfb-host-run-netns\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.971207 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.970575    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4a99fb8d-bf83-4929-8838-c1649ba58cfb-host-run-netns\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.971207 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.970594    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a99fb8d-bf83-4929-8838-c1649ba58cfb-run-openvswitch\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.971207 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.970626    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4a99fb8d-bf83-4929-8838-c1649ba58cfb-host-cni-netd\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.971207 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.970649    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e44079c-1f38-4216-bb3b-9bee9ce4338d-system-cni-dir\") pod \"multus-ls9b9\" (UID: \"5e44079c-1f38-4216-bb3b-9bee9ce4338d\") " pod="openshift-multus/multus-ls9b9"
Jul 07 17:07:58.971207 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.970671    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5e44079c-1f38-4216-bb3b-9bee9ce4338d-os-release\") pod \"multus-ls9b9\" (UID: \"5e44079c-1f38-4216-bb3b-9bee9ce4338d\") " pod="openshift-multus/multus-ls9b9"
Jul 07 17:07:58.971207 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.970678    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4a99fb8d-bf83-4929-8838-c1649ba58cfb-host-cni-netd\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.971207 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.970682    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a99fb8d-bf83-4929-8838-c1649ba58cfb-run-openvswitch\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.971207 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.970689    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5e44079c-1f38-4216-bb3b-9bee9ce4338d-cni-binary-copy\") pod \"multus-ls9b9\" (UID: \"5e44079c-1f38-4216-bb3b-9bee9ce4338d\") " pod="openshift-multus/multus-ls9b9"
Jul 07 17:07:58.971207 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.970733    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/66c074da-39ba-4e2e-ada6-9f71c051f3ec-socket-dir\") pod \"aws-ebs-csi-driver-node-vld2l\" (UID: \"66c074da-39ba-4e2e-ada6-9f71c051f3ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vld2l"
Jul 07 17:07:58.971207 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.970764    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4a99fb8d-bf83-4929-8838-c1649ba58cfb-log-socket\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.971207 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.970790    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fc4d0910-9984-483e-9079-22b47c0defdb-konnectivity-ca\") pod \"konnectivity-agent-ktm2t\" (UID: \"fc4d0910-9984-483e-9079-22b47c0defdb\") " pod="kube-system/konnectivity-agent-ktm2t"
Jul 07 17:07:58.971207 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.970820    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5e44079c-1f38-4216-bb3b-9bee9ce4338d-host-run-k8s-cni-cncf-io\") pod \"multus-ls9b9\" (UID: \"5e44079c-1f38-4216-bb3b-9bee9ce4338d\") " pod="openshift-multus/multus-ls9b9"
Jul 07 17:07:58.971207 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.970829    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4a99fb8d-bf83-4929-8838-c1649ba58cfb-log-socket\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.971207 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.970848    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/66c074da-39ba-4e2e-ada6-9f71c051f3ec-device-dir\") pod \"aws-ebs-csi-driver-node-vld2l\" (UID: \"66c074da-39ba-4e2e-ada6-9f71c051f3ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vld2l"
Jul 07 17:07:58.971207 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.970880    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/49ea513d-8038-4a64-923b-5a32649fa089-iptables-alerter-script\") pod \"iptables-alerter-j6qcn\" (UID: \"49ea513d-8038-4a64-923b-5a32649fa089\") " pod="openshift-network-operator/iptables-alerter-j6qcn"
Jul 07 17:07:58.971207 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.970928    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fc4d0910-9984-483e-9079-22b47c0defdb-agent-certs\") pod \"konnectivity-agent-ktm2t\" (UID: \"fc4d0910-9984-483e-9079-22b47c0defdb\") " pod="kube-system/konnectivity-agent-ktm2t"
Jul 07 17:07:58.971996 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.970953    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5e44079c-1f38-4216-bb3b-9bee9ce4338d-multus-conf-dir\") pod \"multus-ls9b9\" (UID: \"5e44079c-1f38-4216-bb3b-9bee9ce4338d\") " pod="openshift-multus/multus-ls9b9"
Jul 07 17:07:58.971996 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.970979    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58d4x\" (UniqueName: \"kubernetes.io/projected/5e44079c-1f38-4216-bb3b-9bee9ce4338d-kube-api-access-58d4x\") pod \"multus-ls9b9\" (UID: \"5e44079c-1f38-4216-bb3b-9bee9ce4338d\") " pod="openshift-multus/multus-ls9b9"
Jul 07 17:07:58.971996 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.971003    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5d706921-ddd7-4b39-840d-a03cef437d64-kubelet-config\") pod \"global-pull-secret-syncer-wqh28\" (UID: \"5d706921-ddd7-4b39-840d-a03cef437d64\") " pod="kube-system/global-pull-secret-syncer-wqh28"
Jul 07 17:07:58.971996 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.971022    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1c55e58a-1b4a-4643-923c-c235327231ed-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4h5b6\" (UID: \"1c55e58a-1b4a-4643-923c-c235327231ed\") " pod="openshift-multus/multus-additional-cni-plugins-4h5b6"
Jul 07 17:07:58.971996 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.971037    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a4018c2b-fdcb-454a-b094-c2ed4e08e8ef-etc-sysctl-conf\") pod \"tuned-2mpbm\" (UID: \"a4018c2b-fdcb-454a-b094-c2ed4e08e8ef\") " pod="openshift-cluster-node-tuning-operator/tuned-2mpbm"
Jul 07 17:07:58.971996 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.971053    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a4018c2b-fdcb-454a-b094-c2ed4e08e8ef-sys\") pod \"tuned-2mpbm\" (UID: \"a4018c2b-fdcb-454a-b094-c2ed4e08e8ef\") " pod="openshift-cluster-node-tuning-operator/tuned-2mpbm"
Jul 07 17:07:58.971996 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.971070    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctvdc\" (UniqueName: \"kubernetes.io/projected/a4018c2b-fdcb-454a-b094-c2ed4e08e8ef-kube-api-access-ctvdc\") pod \"tuned-2mpbm\" (UID: \"a4018c2b-fdcb-454a-b094-c2ed4e08e8ef\") " pod="openshift-cluster-node-tuning-operator/tuned-2mpbm"
Jul 07 17:07:58.971996 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.971087    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5e44079c-1f38-4216-bb3b-9bee9ce4338d-multus-socket-dir-parent\") pod \"multus-ls9b9\" (UID: \"5e44079c-1f38-4216-bb3b-9bee9ce4338d\") " pod="openshift-multus/multus-ls9b9"
Jul 07 17:07:58.971996 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.971102    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5e44079c-1f38-4216-bb3b-9bee9ce4338d-host-var-lib-cni-bin\") pod \"multus-ls9b9\" (UID: \"5e44079c-1f38-4216-bb3b-9bee9ce4338d\") " pod="openshift-multus/multus-ls9b9"
Jul 07 17:07:58.971996 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.971117    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e44079c-1f38-4216-bb3b-9bee9ce4338d-host-var-lib-kubelet\") pod \"multus-ls9b9\" (UID: \"5e44079c-1f38-4216-bb3b-9bee9ce4338d\") " pod="openshift-multus/multus-ls9b9"
Jul 07 17:07:58.971996 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.971156    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4a99fb8d-bf83-4929-8838-c1649ba58cfb-run-ovn\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.971996 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.971186    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4a99fb8d-bf83-4929-8838-c1649ba58cfb-host-cni-bin\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.971996 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.971196    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4a99fb8d-bf83-4929-8838-c1649ba58cfb-run-ovn\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.971996 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.971215    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbv8k\" (UniqueName: \"kubernetes.io/projected/d6844e3d-9b85-4d30-86db-35466ed9a36e-kube-api-access-jbv8k\") pod \"network-check-target-fm4tg\" (UID: \"d6844e3d-9b85-4d30-86db-35466ed9a36e\") " pod="openshift-network-diagnostics/network-check-target-fm4tg"
Jul 07 17:07:58.971996 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.971237    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4a99fb8d-bf83-4929-8838-c1649ba58cfb-host-cni-bin\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.971996 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.971240    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/187a8f3e-75c8-461a-a0f0-f3e3723d3a34-host\") pod \"node-ca-87n7p\" (UID: \"187a8f3e-75c8-461a-a0f0-f3e3723d3a34\") " pod="openshift-image-registry/node-ca-87n7p"
Jul 07 17:07:58.971996 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.971263    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a4018c2b-fdcb-454a-b094-c2ed4e08e8ef-lib-modules\") pod \"tuned-2mpbm\" (UID: \"a4018c2b-fdcb-454a-b094-c2ed4e08e8ef\") " pod="openshift-cluster-node-tuning-operator/tuned-2mpbm"
Jul 07 17:07:58.972658 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.971281    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a4018c2b-fdcb-454a-b094-c2ed4e08e8ef-var-lib-kubelet\") pod \"tuned-2mpbm\" (UID: \"a4018c2b-fdcb-454a-b094-c2ed4e08e8ef\") " pod="openshift-cluster-node-tuning-operator/tuned-2mpbm"
Jul 07 17:07:58.972658 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.971300    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/66c074da-39ba-4e2e-ada6-9f71c051f3ec-etc-selinux\") pod \"aws-ebs-csi-driver-node-vld2l\" (UID: \"66c074da-39ba-4e2e-ada6-9f71c051f3ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vld2l"
Jul 07 17:07:58.972658 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.971324    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5d706921-ddd7-4b39-840d-a03cef437d64-original-pull-secret\") pod \"global-pull-secret-syncer-wqh28\" (UID: \"5d706921-ddd7-4b39-840d-a03cef437d64\") " pod="kube-system/global-pull-secret-syncer-wqh28"
Jul 07 17:07:58.972658 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.971346    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1c55e58a-1b4a-4643-923c-c235327231ed-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4h5b6\" (UID: \"1c55e58a-1b4a-4643-923c-c235327231ed\") " pod="openshift-multus/multus-additional-cni-plugins-4h5b6"
Jul 07 17:07:58.972658 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.971366    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4a99fb8d-bf83-4929-8838-c1649ba58cfb-node-log\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.972658 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.971382    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4a99fb8d-bf83-4929-8838-c1649ba58cfb-ovn-node-metrics-cert\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.972658 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.971397    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c398fff8-ec63-4063-b3a4-92bf0aa5df9c-metrics-certs\") pod \"network-metrics-daemon-24rwk\" (UID: \"c398fff8-ec63-4063-b3a4-92bf0aa5df9c\") " pod="openshift-multus/network-metrics-daemon-24rwk"
Jul 07 17:07:58.972658 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.971412    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/66c074da-39ba-4e2e-ada6-9f71c051f3ec-registration-dir\") pod \"aws-ebs-csi-driver-node-vld2l\" (UID: \"66c074da-39ba-4e2e-ada6-9f71c051f3ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vld2l"
Jul 07 17:07:58.972658 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.971447    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4a99fb8d-bf83-4929-8838-c1649ba58cfb-node-log\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.972658 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.971448    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bk7m9\" (UniqueName: \"kubernetes.io/projected/66c074da-39ba-4e2e-ada6-9f71c051f3ec-kube-api-access-bk7m9\") pod \"aws-ebs-csi-driver-node-vld2l\" (UID: \"66c074da-39ba-4e2e-ada6-9f71c051f3ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vld2l"
Jul 07 17:07:58.972658 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.971483    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1c55e58a-1b4a-4643-923c-c235327231ed-cnibin\") pod \"multus-additional-cni-plugins-4h5b6\" (UID: \"1c55e58a-1b4a-4643-923c-c235327231ed\") " pod="openshift-multus/multus-additional-cni-plugins-4h5b6"
Jul 07 17:07:58.972658 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:07:58.971528    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 17:07:58.972658 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:07:58.971589    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c398fff8-ec63-4063-b3a4-92bf0aa5df9c-metrics-certs podName:c398fff8-ec63-4063-b3a4-92bf0aa5df9c nodeName:}" failed. No retries permitted until 2026-07-07 17:07:59.471572755 +0000 UTC m=+2.967651265 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c398fff8-ec63-4063-b3a4-92bf0aa5df9c-metrics-certs") pod "network-metrics-daemon-24rwk" (UID: "c398fff8-ec63-4063-b3a4-92bf0aa5df9c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 17:07:58.972658 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.971528    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9h4b\" (UniqueName: \"kubernetes.io/projected/1c55e58a-1b4a-4643-923c-c235327231ed-kube-api-access-q9h4b\") pod \"multus-additional-cni-plugins-4h5b6\" (UID: \"1c55e58a-1b4a-4643-923c-c235327231ed\") " pod="openshift-multus/multus-additional-cni-plugins-4h5b6"
Jul 07 17:07:58.972658 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.971614    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a99fb8d-bf83-4929-8838-c1649ba58cfb-host-kubelet\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.972658 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.971633    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a99fb8d-bf83-4929-8838-c1649ba58cfb-host-run-ovn-kubernetes\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.973162 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.971649    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4a99fb8d-bf83-4929-8838-c1649ba58cfb-env-overrides\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.973162 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.971665    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a4018c2b-fdcb-454a-b094-c2ed4e08e8ef-etc-tuned\") pod \"tuned-2mpbm\" (UID: \"a4018c2b-fdcb-454a-b094-c2ed4e08e8ef\") " pod="openshift-cluster-node-tuning-operator/tuned-2mpbm"
Jul 07 17:07:58.973162 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.971681    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ecf1ac63-1ea5-4590-8d60-b199e5eff959-hosts-file\") pod \"node-resolver-66zv2\" (UID: \"ecf1ac63-1ea5-4590-8d60-b199e5eff959\") " pod="openshift-dns/node-resolver-66zv2"
Jul 07 17:07:58.973162 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.971697    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/66c074da-39ba-4e2e-ada6-9f71c051f3ec-sys-fs\") pod \"aws-ebs-csi-driver-node-vld2l\" (UID: \"66c074da-39ba-4e2e-ada6-9f71c051f3ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vld2l"
Jul 07 17:07:58.973162 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.971711    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a99fb8d-bf83-4929-8838-c1649ba58cfb-host-run-ovn-kubernetes\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.973162 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.971723    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4a99fb8d-bf83-4929-8838-c1649ba58cfb-host-slash\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.973162 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.971739    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a99fb8d-bf83-4929-8838-c1649ba58cfb-host-kubelet\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.973162 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.971812    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4a99fb8d-bf83-4929-8838-c1649ba58cfb-host-slash\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.973162 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.971871    2579 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 07 17:07:58.973162 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.972129    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4a99fb8d-bf83-4929-8838-c1649ba58cfb-env-overrides\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.975567 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.975544    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4a99fb8d-bf83-4929-8838-c1649ba58cfb-ovn-node-metrics-cert\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:58.995441 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:07:58.995406    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 17:07:58.995441 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:07:58.995441    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 17:07:58.995635 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:07:58.995457    2579 projected.go:196] Error preparing data for projected volume kube-api-access-jbv8k for pod openshift-network-diagnostics/network-check-target-fm4tg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 17:07:58.995635 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:07:58.995537    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d6844e3d-9b85-4d30-86db-35466ed9a36e-kube-api-access-jbv8k podName:d6844e3d-9b85-4d30-86db-35466ed9a36e nodeName:}" failed. No retries permitted until 2026-07-07 17:07:59.495506083 +0000 UTC m=+2.991584585 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-jbv8k" (UniqueName: "kubernetes.io/projected/d6844e3d-9b85-4d30-86db-35466ed9a36e-kube-api-access-jbv8k") pod "network-check-target-fm4tg" (UID: "d6844e3d-9b85-4d30-86db-35466ed9a36e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 17:07:58.996790 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:58.996767    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m56zp\" (UniqueName: \"kubernetes.io/projected/c398fff8-ec63-4063-b3a4-92bf0aa5df9c-kube-api-access-m56zp\") pod \"network-metrics-daemon-24rwk\" (UID: \"c398fff8-ec63-4063-b3a4-92bf0aa5df9c\") " pod="openshift-multus/network-metrics-daemon-24rwk"
Jul 07 17:07:59.001374 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.001351    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r54vz\" (UniqueName: \"kubernetes.io/projected/4a99fb8d-bf83-4929-8838-c1649ba58cfb-kube-api-access-r54vz\") pod \"ovnkube-node-vhsvz\" (UID: \"4a99fb8d-bf83-4929-8838-c1649ba58cfb\") " pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:59.026656 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.026603    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-103.ec2.internal" event={"ID":"3df4ea321bf474671e41e2422091c862","Type":"ContainerStarted","Data":"58de8a8e97f0534db9608d34a8e7714172c9ab72e902c6752c0e9f4bd2668c3f"}
Jul 07 17:07:59.027700 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.027674    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-103.ec2.internal" event={"ID":"c3ab2f1cccd249a4a2fa498a1527f9d2","Type":"ContainerStarted","Data":"9bfb71c96bcdb6918dae7bf607606cd2e9444a215594f418404dc60514780466"}
Jul 07 17:07:59.072592 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.072554    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/66c074da-39ba-4e2e-ada6-9f71c051f3ec-device-dir\") pod \"aws-ebs-csi-driver-node-vld2l\" (UID: \"66c074da-39ba-4e2e-ada6-9f71c051f3ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vld2l"
Jul 07 17:07:59.072592 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.072593    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/49ea513d-8038-4a64-923b-5a32649fa089-iptables-alerter-script\") pod \"iptables-alerter-j6qcn\" (UID: \"49ea513d-8038-4a64-923b-5a32649fa089\") " pod="openshift-network-operator/iptables-alerter-j6qcn"
Jul 07 17:07:59.072843 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.072632    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fc4d0910-9984-483e-9079-22b47c0defdb-agent-certs\") pod \"konnectivity-agent-ktm2t\" (UID: \"fc4d0910-9984-483e-9079-22b47c0defdb\") " pod="kube-system/konnectivity-agent-ktm2t"
Jul 07 17:07:59.072843 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.072659    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5e44079c-1f38-4216-bb3b-9bee9ce4338d-multus-conf-dir\") pod \"multus-ls9b9\" (UID: \"5e44079c-1f38-4216-bb3b-9bee9ce4338d\") " pod="openshift-multus/multus-ls9b9"
Jul 07 17:07:59.072843 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.072684    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58d4x\" (UniqueName: \"kubernetes.io/projected/5e44079c-1f38-4216-bb3b-9bee9ce4338d-kube-api-access-58d4x\") pod \"multus-ls9b9\" (UID: \"5e44079c-1f38-4216-bb3b-9bee9ce4338d\") " pod="openshift-multus/multus-ls9b9"
Jul 07 17:07:59.072843 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.072697    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/66c074da-39ba-4e2e-ada6-9f71c051f3ec-device-dir\") pod \"aws-ebs-csi-driver-node-vld2l\" (UID: \"66c074da-39ba-4e2e-ada6-9f71c051f3ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vld2l"
Jul 07 17:07:59.072843 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.072710    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5d706921-ddd7-4b39-840d-a03cef437d64-kubelet-config\") pod \"global-pull-secret-syncer-wqh28\" (UID: \"5d706921-ddd7-4b39-840d-a03cef437d64\") " pod="kube-system/global-pull-secret-syncer-wqh28"
Jul 07 17:07:59.072843 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.072750    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1c55e58a-1b4a-4643-923c-c235327231ed-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4h5b6\" (UID: \"1c55e58a-1b4a-4643-923c-c235327231ed\") " pod="openshift-multus/multus-additional-cni-plugins-4h5b6"
Jul 07 17:07:59.072843 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.072763    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5d706921-ddd7-4b39-840d-a03cef437d64-kubelet-config\") pod \"global-pull-secret-syncer-wqh28\" (UID: \"5d706921-ddd7-4b39-840d-a03cef437d64\") " pod="kube-system/global-pull-secret-syncer-wqh28"
Jul 07 17:07:59.072843 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.072782    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a4018c2b-fdcb-454a-b094-c2ed4e08e8ef-etc-sysctl-conf\") pod \"tuned-2mpbm\" (UID: \"a4018c2b-fdcb-454a-b094-c2ed4e08e8ef\") " pod="openshift-cluster-node-tuning-operator/tuned-2mpbm"
Jul 07 17:07:59.072843 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.072807    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a4018c2b-fdcb-454a-b094-c2ed4e08e8ef-sys\") pod \"tuned-2mpbm\" (UID: \"a4018c2b-fdcb-454a-b094-c2ed4e08e8ef\") " pod="openshift-cluster-node-tuning-operator/tuned-2mpbm"
Jul 07 17:07:59.072843 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.072831    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctvdc\" (UniqueName: \"kubernetes.io/projected/a4018c2b-fdcb-454a-b094-c2ed4e08e8ef-kube-api-access-ctvdc\") pod \"tuned-2mpbm\" (UID: \"a4018c2b-fdcb-454a-b094-c2ed4e08e8ef\") " pod="openshift-cluster-node-tuning-operator/tuned-2mpbm"
Jul 07 17:07:59.073395 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.072856    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5e44079c-1f38-4216-bb3b-9bee9ce4338d-multus-socket-dir-parent\") pod \"multus-ls9b9\" (UID: \"5e44079c-1f38-4216-bb3b-9bee9ce4338d\") " pod="openshift-multus/multus-ls9b9"
Jul 07 17:07:59.073395 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.072880    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5e44079c-1f38-4216-bb3b-9bee9ce4338d-host-var-lib-cni-bin\") pod \"multus-ls9b9\" (UID: \"5e44079c-1f38-4216-bb3b-9bee9ce4338d\") " pod="openshift-multus/multus-ls9b9"
Jul 07 17:07:59.073395 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.072925    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e44079c-1f38-4216-bb3b-9bee9ce4338d-host-var-lib-kubelet\") pod \"multus-ls9b9\" (UID: \"5e44079c-1f38-4216-bb3b-9bee9ce4338d\") " pod="openshift-multus/multus-ls9b9"
Jul 07 17:07:59.073395 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.072990    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/187a8f3e-75c8-461a-a0f0-f3e3723d3a34-host\") pod \"node-ca-87n7p\" (UID: \"187a8f3e-75c8-461a-a0f0-f3e3723d3a34\") " pod="openshift-image-registry/node-ca-87n7p"
Jul 07 17:07:59.073395 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.073017    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a4018c2b-fdcb-454a-b094-c2ed4e08e8ef-lib-modules\") pod \"tuned-2mpbm\" (UID: \"a4018c2b-fdcb-454a-b094-c2ed4e08e8ef\") " pod="openshift-cluster-node-tuning-operator/tuned-2mpbm"
Jul 07 17:07:59.073395 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.073039    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a4018c2b-fdcb-454a-b094-c2ed4e08e8ef-var-lib-kubelet\") pod \"tuned-2mpbm\" (UID: \"a4018c2b-fdcb-454a-b094-c2ed4e08e8ef\") " pod="openshift-cluster-node-tuning-operator/tuned-2mpbm"
Jul 07 17:07:59.073395 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.073065    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/66c074da-39ba-4e2e-ada6-9f71c051f3ec-etc-selinux\") pod \"aws-ebs-csi-driver-node-vld2l\" (UID: \"66c074da-39ba-4e2e-ada6-9f71c051f3ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vld2l"
Jul 07 17:07:59.073395 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.073092    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5d706921-ddd7-4b39-840d-a03cef437d64-original-pull-secret\") pod \"global-pull-secret-syncer-wqh28\" (UID: \"5d706921-ddd7-4b39-840d-a03cef437d64\") " pod="kube-system/global-pull-secret-syncer-wqh28"
Jul 07 17:07:59.073395 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.073121    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1c55e58a-1b4a-4643-923c-c235327231ed-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4h5b6\" (UID: \"1c55e58a-1b4a-4643-923c-c235327231ed\") " pod="openshift-multus/multus-additional-cni-plugins-4h5b6"
Jul 07 17:07:59.073395 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.073163    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/66c074da-39ba-4e2e-ada6-9f71c051f3ec-registration-dir\") pod \"aws-ebs-csi-driver-node-vld2l\" (UID: \"66c074da-39ba-4e2e-ada6-9f71c051f3ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vld2l"
Jul 07 17:07:59.073395 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.073189    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bk7m9\" (UniqueName: \"kubernetes.io/projected/66c074da-39ba-4e2e-ada6-9f71c051f3ec-kube-api-access-bk7m9\") pod \"aws-ebs-csi-driver-node-vld2l\" (UID: \"66c074da-39ba-4e2e-ada6-9f71c051f3ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vld2l"
Jul 07 17:07:59.073395 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.073213    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1c55e58a-1b4a-4643-923c-c235327231ed-cnibin\") pod \"multus-additional-cni-plugins-4h5b6\" (UID: \"1c55e58a-1b4a-4643-923c-c235327231ed\") " pod="openshift-multus/multus-additional-cni-plugins-4h5b6"
Jul 07 17:07:59.073395 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.073241    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9h4b\" (UniqueName: \"kubernetes.io/projected/1c55e58a-1b4a-4643-923c-c235327231ed-kube-api-access-q9h4b\") pod \"multus-additional-cni-plugins-4h5b6\" (UID: \"1c55e58a-1b4a-4643-923c-c235327231ed\") " pod="openshift-multus/multus-additional-cni-plugins-4h5b6"
Jul 07 17:07:59.073395 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.073254    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/187a8f3e-75c8-461a-a0f0-f3e3723d3a34-host\") pod \"node-ca-87n7p\" (UID: \"187a8f3e-75c8-461a-a0f0-f3e3723d3a34\") " pod="openshift-image-registry/node-ca-87n7p"
Jul 07 17:07:59.073395 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.073270    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a4018c2b-fdcb-454a-b094-c2ed4e08e8ef-etc-tuned\") pod \"tuned-2mpbm\" (UID: \"a4018c2b-fdcb-454a-b094-c2ed4e08e8ef\") " pod="openshift-cluster-node-tuning-operator/tuned-2mpbm"
Jul 07 17:07:59.073395 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.073297    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ecf1ac63-1ea5-4590-8d60-b199e5eff959-hosts-file\") pod \"node-resolver-66zv2\" (UID: \"ecf1ac63-1ea5-4590-8d60-b199e5eff959\") " pod="openshift-dns/node-resolver-66zv2"
Jul 07 17:07:59.073395 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.073322    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/66c074da-39ba-4e2e-ada6-9f71c051f3ec-sys-fs\") pod \"aws-ebs-csi-driver-node-vld2l\" (UID: \"66c074da-39ba-4e2e-ada6-9f71c051f3ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vld2l"
Jul 07 17:07:59.074267 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.073358    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a4018c2b-fdcb-454a-b094-c2ed4e08e8ef-run\") pod \"tuned-2mpbm\" (UID: \"a4018c2b-fdcb-454a-b094-c2ed4e08e8ef\") " pod="openshift-cluster-node-tuning-operator/tuned-2mpbm"
Jul 07 17:07:59.074267 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.073382    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5e44079c-1f38-4216-bb3b-9bee9ce4338d-host-run-netns\") pod \"multus-ls9b9\" (UID: \"5e44079c-1f38-4216-bb3b-9bee9ce4338d\") " pod="openshift-multus/multus-ls9b9"
Jul 07 17:07:59.074267 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.073408    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5e44079c-1f38-4216-bb3b-9bee9ce4338d-host-var-lib-cni-multus\") pod \"multus-ls9b9\" (UID: \"5e44079c-1f38-4216-bb3b-9bee9ce4338d\") " pod="openshift-multus/multus-ls9b9"
Jul 07 17:07:59.074267 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.073433    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5e44079c-1f38-4216-bb3b-9bee9ce4338d-etc-kubernetes\") pod \"multus-ls9b9\" (UID: \"5e44079c-1f38-4216-bb3b-9bee9ce4338d\") " pod="openshift-multus/multus-ls9b9"
Jul 07 17:07:59.074267 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.073471    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sf9t2\" (UniqueName: \"kubernetes.io/projected/187a8f3e-75c8-461a-a0f0-f3e3723d3a34-kube-api-access-sf9t2\") pod \"node-ca-87n7p\" (UID: \"187a8f3e-75c8-461a-a0f0-f3e3723d3a34\") " pod="openshift-image-registry/node-ca-87n7p"
Jul 07 17:07:59.074267 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.073496    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a4018c2b-fdcb-454a-b094-c2ed4e08e8ef-etc-modprobe-d\") pod \"tuned-2mpbm\" (UID: \"a4018c2b-fdcb-454a-b094-c2ed4e08e8ef\") " pod="openshift-cluster-node-tuning-operator/tuned-2mpbm"
Jul 07 17:07:59.074267 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.073521    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a4018c2b-fdcb-454a-b094-c2ed4e08e8ef-host\") pod \"tuned-2mpbm\" (UID: \"a4018c2b-fdcb-454a-b094-c2ed4e08e8ef\") " pod="openshift-cluster-node-tuning-operator/tuned-2mpbm"
Jul 07 17:07:59.074267 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.073596    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ecf1ac63-1ea5-4590-8d60-b199e5eff959-tmp-dir\") pod \"node-resolver-66zv2\" (UID: \"ecf1ac63-1ea5-4590-8d60-b199e5eff959\") " pod="openshift-dns/node-resolver-66zv2"
Jul 07 17:07:59.074267 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.073627    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7cnj\" (UniqueName: \"kubernetes.io/projected/ecf1ac63-1ea5-4590-8d60-b199e5eff959-kube-api-access-x7cnj\") pod \"node-resolver-66zv2\" (UID: \"ecf1ac63-1ea5-4590-8d60-b199e5eff959\") " pod="openshift-dns/node-resolver-66zv2"
Jul 07 17:07:59.074267 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.073658    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1c55e58a-1b4a-4643-923c-c235327231ed-system-cni-dir\") pod \"multus-additional-cni-plugins-4h5b6\" (UID: \"1c55e58a-1b4a-4643-923c-c235327231ed\") " pod="openshift-multus/multus-additional-cni-plugins-4h5b6"
Jul 07 17:07:59.074267 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.073687    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1c55e58a-1b4a-4643-923c-c235327231ed-cni-binary-copy\") pod \"multus-additional-cni-plugins-4h5b6\" (UID: \"1c55e58a-1b4a-4643-923c-c235327231ed\") " pod="openshift-multus/multus-additional-cni-plugins-4h5b6"
Jul 07 17:07:59.074267 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.073717    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a4018c2b-fdcb-454a-b094-c2ed4e08e8ef-tmp\") pod \"tuned-2mpbm\" (UID: \"a4018c2b-fdcb-454a-b094-c2ed4e08e8ef\") " pod="openshift-cluster-node-tuning-operator/tuned-2mpbm"
Jul 07 17:07:59.074267 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.073764    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5e44079c-1f38-4216-bb3b-9bee9ce4338d-cnibin\") pod \"multus-ls9b9\" (UID: \"5e44079c-1f38-4216-bb3b-9bee9ce4338d\") " pod="openshift-multus/multus-ls9b9"
Jul 07 17:07:59.074267 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.073798    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5e44079c-1f38-4216-bb3b-9bee9ce4338d-hostroot\") pod \"multus-ls9b9\" (UID: \"5e44079c-1f38-4216-bb3b-9bee9ce4338d\") " pod="openshift-multus/multus-ls9b9"
Jul 07 17:07:59.074267 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.073841    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5e44079c-1f38-4216-bb3b-9bee9ce4338d-multus-daemon-config\") pod \"multus-ls9b9\" (UID: \"5e44079c-1f38-4216-bb3b-9bee9ce4338d\") " pod="openshift-multus/multus-ls9b9"
Jul 07 17:07:59.074267 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.073841    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1c55e58a-1b4a-4643-923c-c235327231ed-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4h5b6\" (UID: \"1c55e58a-1b4a-4643-923c-c235327231ed\") " pod="openshift-multus/multus-additional-cni-plugins-4h5b6"
Jul 07 17:07:59.074267 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.073869    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/49ea513d-8038-4a64-923b-5a32649fa089-host-slash\") pod \"iptables-alerter-j6qcn\" (UID: \"49ea513d-8038-4a64-923b-5a32649fa089\") " pod="openshift-network-operator/iptables-alerter-j6qcn"
Jul 07 17:07:59.074267 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.073896    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1c55e58a-1b4a-4643-923c-c235327231ed-os-release\") pod \"multus-additional-cni-plugins-4h5b6\" (UID: \"1c55e58a-1b4a-4643-923c-c235327231ed\") " pod="openshift-multus/multus-additional-cni-plugins-4h5b6"
Jul 07 17:07:59.075193 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.073927    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/66c074da-39ba-4e2e-ada6-9f71c051f3ec-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vld2l\" (UID: \"66c074da-39ba-4e2e-ada6-9f71c051f3ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vld2l"
Jul 07 17:07:59.075193 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.073953    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5d706921-ddd7-4b39-840d-a03cef437d64-dbus\") pod \"global-pull-secret-syncer-wqh28\" (UID: \"5d706921-ddd7-4b39-840d-a03cef437d64\") " pod="kube-system/global-pull-secret-syncer-wqh28"
Jul 07 17:07:59.075193 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.073989    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5e44079c-1f38-4216-bb3b-9bee9ce4338d-host-var-lib-cni-bin\") pod \"multus-ls9b9\" (UID: \"5e44079c-1f38-4216-bb3b-9bee9ce4338d\") " pod="openshift-multus/multus-ls9b9"
Jul 07 17:07:59.075193 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.074003    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a4018c2b-fdcb-454a-b094-c2ed4e08e8ef-etc-kubernetes\") pod \"tuned-2mpbm\" (UID: \"a4018c2b-fdcb-454a-b094-c2ed4e08e8ef\") " pod="openshift-cluster-node-tuning-operator/tuned-2mpbm"
Jul 07 17:07:59.075193 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.074040    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e44079c-1f38-4216-bb3b-9bee9ce4338d-host-var-lib-kubelet\") pod \"multus-ls9b9\" (UID: \"5e44079c-1f38-4216-bb3b-9bee9ce4338d\") " pod="openshift-multus/multus-ls9b9"
Jul 07 17:07:59.075193 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.074047    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e44079c-1f38-4216-bb3b-9bee9ce4338d-multus-cni-dir\") pod \"multus-ls9b9\" (UID: \"5e44079c-1f38-4216-bb3b-9bee9ce4338d\") " pod="openshift-multus/multus-ls9b9"
Jul 07 17:07:59.075193 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.074056    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a4018c2b-fdcb-454a-b094-c2ed4e08e8ef-etc-sysctl-conf\") pod \"tuned-2mpbm\" (UID: \"a4018c2b-fdcb-454a-b094-c2ed4e08e8ef\") " pod="openshift-cluster-node-tuning-operator/tuned-2mpbm"
Jul 07 17:07:59.075193 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.074077    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/187a8f3e-75c8-461a-a0f0-f3e3723d3a34-serviceca\") pod \"node-ca-87n7p\" (UID: \"187a8f3e-75c8-461a-a0f0-f3e3723d3a34\") " pod="openshift-image-registry/node-ca-87n7p"
Jul 07 17:07:59.075193 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.074134    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a4018c2b-fdcb-454a-b094-c2ed4e08e8ef-etc-systemd\") pod \"tuned-2mpbm\" (UID: \"a4018c2b-fdcb-454a-b094-c2ed4e08e8ef\") " pod="openshift-cluster-node-tuning-operator/tuned-2mpbm"
Jul 07 17:07:59.075193 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.074168    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6fd6\" (UniqueName: \"kubernetes.io/projected/49ea513d-8038-4a64-923b-5a32649fa089-kube-api-access-g6fd6\") pod \"iptables-alerter-j6qcn\" (UID: \"49ea513d-8038-4a64-923b-5a32649fa089\") " pod="openshift-network-operator/iptables-alerter-j6qcn"
Jul 07 17:07:59.075193 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.074202    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1c55e58a-1b4a-4643-923c-c235327231ed-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4h5b6\" (UID: \"1c55e58a-1b4a-4643-923c-c235327231ed\") " pod="openshift-multus/multus-additional-cni-plugins-4h5b6"
Jul 07 17:07:59.075193 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.074234    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a4018c2b-fdcb-454a-b094-c2ed4e08e8ef-etc-sysconfig\") pod \"tuned-2mpbm\" (UID: \"a4018c2b-fdcb-454a-b094-c2ed4e08e8ef\") " pod="openshift-cluster-node-tuning-operator/tuned-2mpbm"
Jul 07 17:07:59.075193 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.074259    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a4018c2b-fdcb-454a-b094-c2ed4e08e8ef-etc-sysctl-d\") pod \"tuned-2mpbm\" (UID: \"a4018c2b-fdcb-454a-b094-c2ed4e08e8ef\") " pod="openshift-cluster-node-tuning-operator/tuned-2mpbm"
Jul 07 17:07:59.075193 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.074286    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5e44079c-1f38-4216-bb3b-9bee9ce4338d-host-run-multus-certs\") pod \"multus-ls9b9\" (UID: \"5e44079c-1f38-4216-bb3b-9bee9ce4338d\") " pod="openshift-multus/multus-ls9b9"
Jul 07 17:07:59.075193 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.074322    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e44079c-1f38-4216-bb3b-9bee9ce4338d-system-cni-dir\") pod \"multus-ls9b9\" (UID: \"5e44079c-1f38-4216-bb3b-9bee9ce4338d\") " pod="openshift-multus/multus-ls9b9"
Jul 07 17:07:59.075193 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.074349    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5e44079c-1f38-4216-bb3b-9bee9ce4338d-os-release\") pod \"multus-ls9b9\" (UID: \"5e44079c-1f38-4216-bb3b-9bee9ce4338d\") " pod="openshift-multus/multus-ls9b9"
Jul 07 17:07:59.075193 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.074374    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5e44079c-1f38-4216-bb3b-9bee9ce4338d-cni-binary-copy\") pod \"multus-ls9b9\" (UID: \"5e44079c-1f38-4216-bb3b-9bee9ce4338d\") " pod="openshift-multus/multus-ls9b9"
Jul 07 17:07:59.076017 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.074402    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/66c074da-39ba-4e2e-ada6-9f71c051f3ec-socket-dir\") pod \"aws-ebs-csi-driver-node-vld2l\" (UID: \"66c074da-39ba-4e2e-ada6-9f71c051f3ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vld2l"
Jul 07 17:07:59.076017 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.074430    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fc4d0910-9984-483e-9079-22b47c0defdb-konnectivity-ca\") pod \"konnectivity-agent-ktm2t\" (UID: \"fc4d0910-9984-483e-9079-22b47c0defdb\") " pod="kube-system/konnectivity-agent-ktm2t"
Jul 07 17:07:59.076017 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.074455    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5e44079c-1f38-4216-bb3b-9bee9ce4338d-host-run-k8s-cni-cncf-io\") pod \"multus-ls9b9\" (UID: \"5e44079c-1f38-4216-bb3b-9bee9ce4338d\") " pod="openshift-multus/multus-ls9b9"
Jul 07 17:07:59.076017 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.074479    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/187a8f3e-75c8-461a-a0f0-f3e3723d3a34-serviceca\") pod \"node-ca-87n7p\" (UID: \"187a8f3e-75c8-461a-a0f0-f3e3723d3a34\") " pod="openshift-image-registry/node-ca-87n7p"
Jul 07 17:07:59.076017 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.074582    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/66c074da-39ba-4e2e-ada6-9f71c051f3ec-registration-dir\") pod \"aws-ebs-csi-driver-node-vld2l\" (UID: \"66c074da-39ba-4e2e-ada6-9f71c051f3ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vld2l"
Jul 07 17:07:59.076017 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.074691    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1c55e58a-1b4a-4643-923c-c235327231ed-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4h5b6\" (UID: \"1c55e58a-1b4a-4643-923c-c235327231ed\") " pod="openshift-multus/multus-additional-cni-plugins-4h5b6"
Jul 07 17:07:59.076017 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.074697    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5e44079c-1f38-4216-bb3b-9bee9ce4338d-multus-conf-dir\") pod \"multus-ls9b9\" (UID: \"5e44079c-1f38-4216-bb3b-9bee9ce4338d\") " pod="openshift-multus/multus-ls9b9"
Jul 07 17:07:59.076017 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.074745    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a4018c2b-fdcb-454a-b094-c2ed4e08e8ef-sys\") pod \"tuned-2mpbm\" (UID: \"a4018c2b-fdcb-454a-b094-c2ed4e08e8ef\") " pod="openshift-cluster-node-tuning-operator/tuned-2mpbm"
Jul 07 17:07:59.076017 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.073926    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5e44079c-1f38-4216-bb3b-9bee9ce4338d-multus-socket-dir-parent\") pod \"multus-ls9b9\" (UID: \"5e44079c-1f38-4216-bb3b-9bee9ce4338d\") " pod="openshift-multus/multus-ls9b9"
Jul 07 17:07:59.076017 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.073428    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/49ea513d-8038-4a64-923b-5a32649fa089-iptables-alerter-script\") pod \"iptables-alerter-j6qcn\" (UID: \"49ea513d-8038-4a64-923b-5a32649fa089\") " pod="openshift-network-operator/iptables-alerter-j6qcn"
Jul 07 17:07:59.076017 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.074953    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a4018c2b-fdcb-454a-b094-c2ed4e08e8ef-run\") pod \"tuned-2mpbm\" (UID: \"a4018c2b-fdcb-454a-b094-c2ed4e08e8ef\") " pod="openshift-cluster-node-tuning-operator/tuned-2mpbm"
Jul 07 17:07:59.076017 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.074990    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a4018c2b-fdcb-454a-b094-c2ed4e08e8ef-etc-systemd\") pod \"tuned-2mpbm\" (UID: \"a4018c2b-fdcb-454a-b094-c2ed4e08e8ef\") " pod="openshift-cluster-node-tuning-operator/tuned-2mpbm"
Jul 07 17:07:59.076017 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.075027    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5e44079c-1f38-4216-bb3b-9bee9ce4338d-host-run-netns\") pod \"multus-ls9b9\" (UID: \"5e44079c-1f38-4216-bb3b-9bee9ce4338d\") " pod="openshift-multus/multus-ls9b9"
Jul 07 17:07:59.076017 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:07:59.075110    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 17:07:59.076017 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:07:59.075172    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d706921-ddd7-4b39-840d-a03cef437d64-original-pull-secret podName:5d706921-ddd7-4b39-840d-a03cef437d64 nodeName:}" failed. No retries permitted until 2026-07-07 17:07:59.575152534 +0000 UTC m=+3.071231037 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5d706921-ddd7-4b39-840d-a03cef437d64-original-pull-secret") pod "global-pull-secret-syncer-wqh28" (UID: "5d706921-ddd7-4b39-840d-a03cef437d64") : object "kube-system"/"original-pull-secret" not registered
Jul 07 17:07:59.076017 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.075191    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ecf1ac63-1ea5-4590-8d60-b199e5eff959-tmp-dir\") pod \"node-resolver-66zv2\" (UID: \"ecf1ac63-1ea5-4590-8d60-b199e5eff959\") " pod="openshift-dns/node-resolver-66zv2"
Jul 07 17:07:59.076017 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.075344    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5d706921-ddd7-4b39-840d-a03cef437d64-dbus\") pod \"global-pull-secret-syncer-wqh28\" (UID: \"5d706921-ddd7-4b39-840d-a03cef437d64\") " pod="kube-system/global-pull-secret-syncer-wqh28"
Jul 07 17:07:59.076917 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.075519    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/49ea513d-8038-4a64-923b-5a32649fa089-host-slash\") pod \"iptables-alerter-j6qcn\" (UID: \"49ea513d-8038-4a64-923b-5a32649fa089\") " pod="openshift-network-operator/iptables-alerter-j6qcn"
Jul 07 17:07:59.076917 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.075544    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5e44079c-1f38-4216-bb3b-9bee9ce4338d-host-var-lib-cni-multus\") pod \"multus-ls9b9\" (UID: \"5e44079c-1f38-4216-bb3b-9bee9ce4338d\") " pod="openshift-multus/multus-ls9b9"
Jul 07 17:07:59.076917 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.075582    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1c55e58a-1b4a-4643-923c-c235327231ed-os-release\") pod \"multus-additional-cni-plugins-4h5b6\" (UID: \"1c55e58a-1b4a-4643-923c-c235327231ed\") " pod="openshift-multus/multus-additional-cni-plugins-4h5b6"
Jul 07 17:07:59.076917 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.075610    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/66c074da-39ba-4e2e-ada6-9f71c051f3ec-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vld2l\" (UID: \"66c074da-39ba-4e2e-ada6-9f71c051f3ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vld2l"
Jul 07 17:07:59.076917 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.075633    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1c55e58a-1b4a-4643-923c-c235327231ed-cnibin\") pod \"multus-additional-cni-plugins-4h5b6\" (UID: \"1c55e58a-1b4a-4643-923c-c235327231ed\") " pod="openshift-multus/multus-additional-cni-plugins-4h5b6"
Jul 07 17:07:59.076917 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.075656    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5e44079c-1f38-4216-bb3b-9bee9ce4338d-etc-kubernetes\") pod \"multus-ls9b9\" (UID: \"5e44079c-1f38-4216-bb3b-9bee9ce4338d\") " pod="openshift-multus/multus-ls9b9"
Jul 07 17:07:59.076917 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.075776    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1c55e58a-1b4a-4643-923c-c235327231ed-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4h5b6\" (UID: \"1c55e58a-1b4a-4643-923c-c235327231ed\") " pod="openshift-multus/multus-additional-cni-plugins-4h5b6"
Jul 07 17:07:59.076917 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.075818    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1c55e58a-1b4a-4643-923c-c235327231ed-system-cni-dir\") pod \"multus-additional-cni-plugins-4h5b6\" (UID: \"1c55e58a-1b4a-4643-923c-c235327231ed\") " pod="openshift-multus/multus-additional-cni-plugins-4h5b6"
Jul 07 17:07:59.076917 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.075850    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a4018c2b-fdcb-454a-b094-c2ed4e08e8ef-etc-sysconfig\") pod \"tuned-2mpbm\" (UID: \"a4018c2b-fdcb-454a-b094-c2ed4e08e8ef\") " pod="openshift-cluster-node-tuning-operator/tuned-2mpbm"
Jul 07 17:07:59.076917 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.075915    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/66c074da-39ba-4e2e-ada6-9f71c051f3ec-sys-fs\") pod \"aws-ebs-csi-driver-node-vld2l\" (UID: \"66c074da-39ba-4e2e-ada6-9f71c051f3ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vld2l"
Jul 07 17:07:59.076917 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.075997    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a4018c2b-fdcb-454a-b094-c2ed4e08e8ef-etc-kubernetes\") pod \"tuned-2mpbm\" (UID: \"a4018c2b-fdcb-454a-b094-c2ed4e08e8ef\") " pod="openshift-cluster-node-tuning-operator/tuned-2mpbm"
Jul 07 17:07:59.076917 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.076032    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fc4d0910-9984-483e-9079-22b47c0defdb-agent-certs\") pod \"konnectivity-agent-ktm2t\" (UID: \"fc4d0910-9984-483e-9079-22b47c0defdb\") " pod="kube-system/konnectivity-agent-ktm2t"
Jul 07 17:07:59.076917 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.076067    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e44079c-1f38-4216-bb3b-9bee9ce4338d-multus-cni-dir\") pod \"multus-ls9b9\" (UID: \"5e44079c-1f38-4216-bb3b-9bee9ce4338d\") " pod="openshift-multus/multus-ls9b9"
Jul 07 17:07:59.076917 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.076076    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a4018c2b-fdcb-454a-b094-c2ed4e08e8ef-host\") pod \"tuned-2mpbm\" (UID: \"a4018c2b-fdcb-454a-b094-c2ed4e08e8ef\") " pod="openshift-cluster-node-tuning-operator/tuned-2mpbm"
Jul 07 17:07:59.076917 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.076112    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ecf1ac63-1ea5-4590-8d60-b199e5eff959-hosts-file\") pod \"node-resolver-66zv2\" (UID: \"ecf1ac63-1ea5-4590-8d60-b199e5eff959\") " pod="openshift-dns/node-resolver-66zv2"
Jul 07 17:07:59.076917 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.076130    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5e44079c-1f38-4216-bb3b-9bee9ce4338d-cnibin\") pod \"multus-ls9b9\" (UID: \"5e44079c-1f38-4216-bb3b-9bee9ce4338d\") " pod="openshift-multus/multus-ls9b9"
Jul 07 17:07:59.076917 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.076159    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a4018c2b-fdcb-454a-b094-c2ed4e08e8ef-etc-modprobe-d\") pod \"tuned-2mpbm\" (UID: \"a4018c2b-fdcb-454a-b094-c2ed4e08e8ef\") " pod="openshift-cluster-node-tuning-operator/tuned-2mpbm"
Jul 07 17:07:59.076917 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.076172    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5e44079c-1f38-4216-bb3b-9bee9ce4338d-host-run-multus-certs\") pod \"multus-ls9b9\" (UID: \"5e44079c-1f38-4216-bb3b-9bee9ce4338d\") " pod="openshift-multus/multus-ls9b9"
Jul 07 17:07:59.077906 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.076215    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e44079c-1f38-4216-bb3b-9bee9ce4338d-system-cni-dir\") pod \"multus-ls9b9\" (UID: \"5e44079c-1f38-4216-bb3b-9bee9ce4338d\") " pod="openshift-multus/multus-ls9b9"
Jul 07 17:07:59.077906 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.076267    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5e44079c-1f38-4216-bb3b-9bee9ce4338d-os-release\") pod \"multus-ls9b9\" (UID: \"5e44079c-1f38-4216-bb3b-9bee9ce4338d\") " pod="openshift-multus/multus-ls9b9"
Jul 07 17:07:59.077906 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.076321    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5e44079c-1f38-4216-bb3b-9bee9ce4338d-multus-daemon-config\") pod \"multus-ls9b9\" (UID: \"5e44079c-1f38-4216-bb3b-9bee9ce4338d\") " pod="openshift-multus/multus-ls9b9"
Jul 07 17:07:59.077906 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.076323    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1c55e58a-1b4a-4643-923c-c235327231ed-cni-binary-copy\") pod \"multus-additional-cni-plugins-4h5b6\" (UID: \"1c55e58a-1b4a-4643-923c-c235327231ed\") " pod="openshift-multus/multus-additional-cni-plugins-4h5b6"
Jul 07 17:07:59.077906 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.076381    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a4018c2b-fdcb-454a-b094-c2ed4e08e8ef-etc-sysctl-d\") pod \"tuned-2mpbm\" (UID: \"a4018c2b-fdcb-454a-b094-c2ed4e08e8ef\") " pod="openshift-cluster-node-tuning-operator/tuned-2mpbm"
Jul 07 17:07:59.077906 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.076440    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5e44079c-1f38-4216-bb3b-9bee9ce4338d-hostroot\") pod \"multus-ls9b9\" (UID: \"5e44079c-1f38-4216-bb3b-9bee9ce4338d\") " pod="openshift-multus/multus-ls9b9"
Jul 07 17:07:59.077906 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.076472    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a4018c2b-fdcb-454a-b094-c2ed4e08e8ef-lib-modules\") pod \"tuned-2mpbm\" (UID: \"a4018c2b-fdcb-454a-b094-c2ed4e08e8ef\") " pod="openshift-cluster-node-tuning-operator/tuned-2mpbm"
Jul 07 17:07:59.077906 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.076517    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5e44079c-1f38-4216-bb3b-9bee9ce4338d-host-run-k8s-cni-cncf-io\") pod \"multus-ls9b9\" (UID: \"5e44079c-1f38-4216-bb3b-9bee9ce4338d\") " pod="openshift-multus/multus-ls9b9"
Jul 07 17:07:59.077906 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.076580    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/66c074da-39ba-4e2e-ada6-9f71c051f3ec-etc-selinux\") pod \"aws-ebs-csi-driver-node-vld2l\" (UID: \"66c074da-39ba-4e2e-ada6-9f71c051f3ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vld2l"
Jul 07 17:07:59.077906 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.076578    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/66c074da-39ba-4e2e-ada6-9f71c051f3ec-socket-dir\") pod \"aws-ebs-csi-driver-node-vld2l\" (UID: \"66c074da-39ba-4e2e-ada6-9f71c051f3ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vld2l"
Jul 07 17:07:59.077906 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.076632    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a4018c2b-fdcb-454a-b094-c2ed4e08e8ef-var-lib-kubelet\") pod \"tuned-2mpbm\" (UID: \"a4018c2b-fdcb-454a-b094-c2ed4e08e8ef\") " pod="openshift-cluster-node-tuning-operator/tuned-2mpbm"
Jul 07 17:07:59.077906 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.076659    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5e44079c-1f38-4216-bb3b-9bee9ce4338d-cni-binary-copy\") pod \"multus-ls9b9\" (UID: \"5e44079c-1f38-4216-bb3b-9bee9ce4338d\") " pod="openshift-multus/multus-ls9b9"
Jul 07 17:07:59.077906 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.077500    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fc4d0910-9984-483e-9079-22b47c0defdb-konnectivity-ca\") pod \"konnectivity-agent-ktm2t\" (UID: \"fc4d0910-9984-483e-9079-22b47c0defdb\") " pod="kube-system/konnectivity-agent-ktm2t"
Jul 07 17:07:59.078753 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.078732    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a4018c2b-fdcb-454a-b094-c2ed4e08e8ef-etc-tuned\") pod \"tuned-2mpbm\" (UID: \"a4018c2b-fdcb-454a-b094-c2ed4e08e8ef\") " pod="openshift-cluster-node-tuning-operator/tuned-2mpbm"
Jul 07 17:07:59.079526 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.079493    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a4018c2b-fdcb-454a-b094-c2ed4e08e8ef-tmp\") pod \"tuned-2mpbm\" (UID: \"a4018c2b-fdcb-454a-b094-c2ed4e08e8ef\") " pod="openshift-cluster-node-tuning-operator/tuned-2mpbm"
Jul 07 17:07:59.103058 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.102980    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctvdc\" (UniqueName: \"kubernetes.io/projected/a4018c2b-fdcb-454a-b094-c2ed4e08e8ef-kube-api-access-ctvdc\") pod \"tuned-2mpbm\" (UID: \"a4018c2b-fdcb-454a-b094-c2ed4e08e8ef\") " pod="openshift-cluster-node-tuning-operator/tuned-2mpbm"
Jul 07 17:07:59.106723 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.106693    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bk7m9\" (UniqueName: \"kubernetes.io/projected/66c074da-39ba-4e2e-ada6-9f71c051f3ec-kube-api-access-bk7m9\") pod \"aws-ebs-csi-driver-node-vld2l\" (UID: \"66c074da-39ba-4e2e-ada6-9f71c051f3ec\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vld2l"
Jul 07 17:07:59.109619 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.109169    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6fd6\" (UniqueName: \"kubernetes.io/projected/49ea513d-8038-4a64-923b-5a32649fa089-kube-api-access-g6fd6\") pod \"iptables-alerter-j6qcn\" (UID: \"49ea513d-8038-4a64-923b-5a32649fa089\") " pod="openshift-network-operator/iptables-alerter-j6qcn"
Jul 07 17:07:59.109619 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.109206    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-58d4x\" (UniqueName: \"kubernetes.io/projected/5e44079c-1f38-4216-bb3b-9bee9ce4338d-kube-api-access-58d4x\") pod \"multus-ls9b9\" (UID: \"5e44079c-1f38-4216-bb3b-9bee9ce4338d\") " pod="openshift-multus/multus-ls9b9"
Jul 07 17:07:59.111762 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.111730    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7cnj\" (UniqueName: \"kubernetes.io/projected/ecf1ac63-1ea5-4590-8d60-b199e5eff959-kube-api-access-x7cnj\") pod \"node-resolver-66zv2\" (UID: \"ecf1ac63-1ea5-4590-8d60-b199e5eff959\") " pod="openshift-dns/node-resolver-66zv2"
Jul 07 17:07:59.113716 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.113100    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sf9t2\" (UniqueName: \"kubernetes.io/projected/187a8f3e-75c8-461a-a0f0-f3e3723d3a34-kube-api-access-sf9t2\") pod \"node-ca-87n7p\" (UID: \"187a8f3e-75c8-461a-a0f0-f3e3723d3a34\") " pod="openshift-image-registry/node-ca-87n7p"
Jul 07 17:07:59.114643 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.114614    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9h4b\" (UniqueName: \"kubernetes.io/projected/1c55e58a-1b4a-4643-923c-c235327231ed-kube-api-access-q9h4b\") pod \"multus-additional-cni-plugins-4h5b6\" (UID: \"1c55e58a-1b4a-4643-923c-c235327231ed\") " pod="openshift-multus/multus-additional-cni-plugins-4h5b6"
Jul 07 17:07:59.168901 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.168845    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:07:59.180396 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.180370    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-66zv2"
Jul 07 17:07:59.203597 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.203561    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-87n7p"
Jul 07 17:07:59.204752 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.204722    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ktm2t"
Jul 07 17:07:59.214765 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.214743    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vld2l"
Jul 07 17:07:59.222983 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.222942    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-j6qcn"
Jul 07 17:07:59.231151 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.231124    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2mpbm"
Jul 07 17:07:59.240220 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.240198    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4h5b6"
Jul 07 17:07:59.247347 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.247326    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ls9b9"
Jul 07 17:07:59.477212 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.477127    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c398fff8-ec63-4063-b3a4-92bf0aa5df9c-metrics-certs\") pod \"network-metrics-daemon-24rwk\" (UID: \"c398fff8-ec63-4063-b3a4-92bf0aa5df9c\") " pod="openshift-multus/network-metrics-daemon-24rwk"
Jul 07 17:07:59.477376 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:07:59.477302    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 17:07:59.477428 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:07:59.477388    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c398fff8-ec63-4063-b3a4-92bf0aa5df9c-metrics-certs podName:c398fff8-ec63-4063-b3a4-92bf0aa5df9c nodeName:}" failed. No retries permitted until 2026-07-07 17:08:00.477364916 +0000 UTC m=+3.973443403 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c398fff8-ec63-4063-b3a4-92bf0aa5df9c-metrics-certs") pod "network-metrics-daemon-24rwk" (UID: "c398fff8-ec63-4063-b3a4-92bf0aa5df9c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 17:07:59.577873 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.577838    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbv8k\" (UniqueName: \"kubernetes.io/projected/d6844e3d-9b85-4d30-86db-35466ed9a36e-kube-api-access-jbv8k\") pod \"network-check-target-fm4tg\" (UID: \"d6844e3d-9b85-4d30-86db-35466ed9a36e\") " pod="openshift-network-diagnostics/network-check-target-fm4tg"
Jul 07 17:07:59.577873 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.577878    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5d706921-ddd7-4b39-840d-a03cef437d64-original-pull-secret\") pod \"global-pull-secret-syncer-wqh28\" (UID: \"5d706921-ddd7-4b39-840d-a03cef437d64\") " pod="kube-system/global-pull-secret-syncer-wqh28"
Jul 07 17:07:59.578117 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:07:59.578020    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 17:07:59.578117 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:07:59.578024    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 17:07:59.578117 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:07:59.578050    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 17:07:59.578117 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:07:59.578060    2579 projected.go:196] Error preparing data for projected volume kube-api-access-jbv8k for pod openshift-network-diagnostics/network-check-target-fm4tg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 17:07:59.578117 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:07:59.578086    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d706921-ddd7-4b39-840d-a03cef437d64-original-pull-secret podName:5d706921-ddd7-4b39-840d-a03cef437d64 nodeName:}" failed. No retries permitted until 2026-07-07 17:08:00.578068553 +0000 UTC m=+4.074147041 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5d706921-ddd7-4b39-840d-a03cef437d64-original-pull-secret") pod "global-pull-secret-syncer-wqh28" (UID: "5d706921-ddd7-4b39-840d-a03cef437d64") : object "kube-system"/"original-pull-secret" not registered
Jul 07 17:07:59.578314 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:07:59.578142    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d6844e3d-9b85-4d30-86db-35466ed9a36e-kube-api-access-jbv8k podName:d6844e3d-9b85-4d30-86db-35466ed9a36e nodeName:}" failed. No retries permitted until 2026-07-07 17:08:00.578102884 +0000 UTC m=+4.074181371 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-jbv8k" (UniqueName: "kubernetes.io/projected/d6844e3d-9b85-4d30-86db-35466ed9a36e-kube-api-access-jbv8k") pod "network-check-target-fm4tg" (UID: "d6844e3d-9b85-4d30-86db-35466ed9a36e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 17:07:59.966630 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.966583    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 17:02:59 +0000 UTC" deadline="2028-02-03 13:28:46.131420504 +0000 UTC"
Jul 07 17:07:59.966630 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:07:59.966624    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13820h20m46.164801218s"
Jul 07 17:08:00.021554 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:00.021524    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fm4tg"
Jul 07 17:08:00.021700 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:00.021642    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fm4tg" podUID="d6844e3d-9b85-4d30-86db-35466ed9a36e"
Jul 07 17:08:00.091910 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:08:00.091671    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e44079c_1f38_4216_bb3b_9bee9ce4338d.slice/crio-02f1f02396bffd103f88c777f56a41d3dba248681777adb98e8510a9dda67d2f WatchSource:0}: Error finding container 02f1f02396bffd103f88c777f56a41d3dba248681777adb98e8510a9dda67d2f: Status 404 returned error can't find the container with id 02f1f02396bffd103f88c777f56a41d3dba248681777adb98e8510a9dda67d2f
Jul 07 17:08:00.096743 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:08:00.096702    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4018c2b_fdcb_454a_b094_c2ed4e08e8ef.slice/crio-6120ed4d89a31ca3b907759fbc24577827d5d5d7f09d09f6b542148996beb4fb WatchSource:0}: Error finding container 6120ed4d89a31ca3b907759fbc24577827d5d5d7f09d09f6b542148996beb4fb: Status 404 returned error can't find the container with id 6120ed4d89a31ca3b907759fbc24577827d5d5d7f09d09f6b542148996beb4fb
Jul 07 17:08:00.099229 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:08:00.099204    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c55e58a_1b4a_4643_923c_c235327231ed.slice/crio-dd235f3508a484de5806f9b3d92a8f70319154b51032c754b72f3051b1283d38 WatchSource:0}: Error finding container dd235f3508a484de5806f9b3d92a8f70319154b51032c754b72f3051b1283d38: Status 404 returned error can't find the container with id dd235f3508a484de5806f9b3d92a8f70319154b51032c754b72f3051b1283d38
Jul 07 17:08:00.100708 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:08:00.100669    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a99fb8d_bf83_4929_8838_c1649ba58cfb.slice/crio-34641d56becceb277cd861f4126e9df5d5e95b6a3f8f489bb0b68eedf87478bf WatchSource:0}: Error finding container 34641d56becceb277cd861f4126e9df5d5e95b6a3f8f489bb0b68eedf87478bf: Status 404 returned error can't find the container with id 34641d56becceb277cd861f4126e9df5d5e95b6a3f8f489bb0b68eedf87478bf
Jul 07 17:08:00.103435 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:08:00.102508    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66c074da_39ba_4e2e_ada6_9f71c051f3ec.slice/crio-126e122aa966753f852f71e4aa3504594dd29ce21ec5e12542ce5c935d6c9974 WatchSource:0}: Error finding container 126e122aa966753f852f71e4aa3504594dd29ce21ec5e12542ce5c935d6c9974: Status 404 returned error can't find the container with id 126e122aa966753f852f71e4aa3504594dd29ce21ec5e12542ce5c935d6c9974
Jul 07 17:08:00.103435 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:08:00.103255    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49ea513d_8038_4a64_923b_5a32649fa089.slice/crio-fd1ead3941d31b6c81f6f5ba08bad4c0f5702739da9f83777782911681dccad5 WatchSource:0}: Error finding container fd1ead3941d31b6c81f6f5ba08bad4c0f5702739da9f83777782911681dccad5: Status 404 returned error can't find the container with id fd1ead3941d31b6c81f6f5ba08bad4c0f5702739da9f83777782911681dccad5
Jul 07 17:08:00.105947 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:08:00.105488    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc4d0910_9984_483e_9079_22b47c0defdb.slice/crio-1fe27bcc6e11db8502fc6e47b9960e56ce58db6e66abc8a8910575762901778c WatchSource:0}: Error finding container 1fe27bcc6e11db8502fc6e47b9960e56ce58db6e66abc8a8910575762901778c: Status 404 returned error can't find the container with id 1fe27bcc6e11db8502fc6e47b9960e56ce58db6e66abc8a8910575762901778c
Jul 07 17:08:00.484663 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:00.484569    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c398fff8-ec63-4063-b3a4-92bf0aa5df9c-metrics-certs\") pod \"network-metrics-daemon-24rwk\" (UID: \"c398fff8-ec63-4063-b3a4-92bf0aa5df9c\") " pod="openshift-multus/network-metrics-daemon-24rwk"
Jul 07 17:08:00.484841 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:00.484820    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 17:08:00.484909 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:00.484895    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c398fff8-ec63-4063-b3a4-92bf0aa5df9c-metrics-certs podName:c398fff8-ec63-4063-b3a4-92bf0aa5df9c nodeName:}" failed. No retries permitted until 2026-07-07 17:08:02.484878749 +0000 UTC m=+5.980957238 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c398fff8-ec63-4063-b3a4-92bf0aa5df9c-metrics-certs") pod "network-metrics-daemon-24rwk" (UID: "c398fff8-ec63-4063-b3a4-92bf0aa5df9c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 17:08:00.586466 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:00.586422    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbv8k\" (UniqueName: \"kubernetes.io/projected/d6844e3d-9b85-4d30-86db-35466ed9a36e-kube-api-access-jbv8k\") pod \"network-check-target-fm4tg\" (UID: \"d6844e3d-9b85-4d30-86db-35466ed9a36e\") " pod="openshift-network-diagnostics/network-check-target-fm4tg"
Jul 07 17:08:00.586659 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:00.586488    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5d706921-ddd7-4b39-840d-a03cef437d64-original-pull-secret\") pod \"global-pull-secret-syncer-wqh28\" (UID: \"5d706921-ddd7-4b39-840d-a03cef437d64\") " pod="kube-system/global-pull-secret-syncer-wqh28"
Jul 07 17:08:00.586731 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:00.586699    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 17:08:00.586813 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:00.586779    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d706921-ddd7-4b39-840d-a03cef437d64-original-pull-secret podName:5d706921-ddd7-4b39-840d-a03cef437d64 nodeName:}" failed. No retries permitted until 2026-07-07 17:08:02.586758426 +0000 UTC m=+6.082836922 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5d706921-ddd7-4b39-840d-a03cef437d64-original-pull-secret") pod "global-pull-secret-syncer-wqh28" (UID: "5d706921-ddd7-4b39-840d-a03cef437d64") : object "kube-system"/"original-pull-secret" not registered
Jul 07 17:08:00.587372 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:00.587350    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 17:08:00.587606 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:00.587385    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 17:08:00.587666 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:00.587627    2579 projected.go:196] Error preparing data for projected volume kube-api-access-jbv8k for pod openshift-network-diagnostics/network-check-target-fm4tg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 17:08:00.587730 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:00.587702    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d6844e3d-9b85-4d30-86db-35466ed9a36e-kube-api-access-jbv8k podName:d6844e3d-9b85-4d30-86db-35466ed9a36e nodeName:}" failed. No retries permitted until 2026-07-07 17:08:02.587676361 +0000 UTC m=+6.083754852 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-jbv8k" (UniqueName: "kubernetes.io/projected/d6844e3d-9b85-4d30-86db-35466ed9a36e-kube-api-access-jbv8k") pod "network-check-target-fm4tg" (UID: "d6844e3d-9b85-4d30-86db-35466ed9a36e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 17:08:00.967499 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:00.967351    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 17:02:59 +0000 UTC" deadline="2028-04-13 11:46:24.91888749 +0000 UTC"
Jul 07 17:08:00.967499 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:00.967434    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15498h38m23.951457536s"
Jul 07 17:08:01.021985 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:01.021932    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqh28"
Jul 07 17:08:01.022184 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:01.022082    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqh28" podUID="5d706921-ddd7-4b39-840d-a03cef437d64"
Jul 07 17:08:01.022610 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:01.022563    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-24rwk"
Jul 07 17:08:01.022707 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:01.022679    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-24rwk" podUID="c398fff8-ec63-4063-b3a4-92bf0aa5df9c"
Jul 07 17:08:01.045689 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:01.045645    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-103.ec2.internal" event={"ID":"3df4ea321bf474671e41e2422091c862","Type":"ContainerStarted","Data":"38f7b25ff1d138953da14e928968a9a4dba802e13974c29705cd27b1f21c57fe"}
Jul 07 17:08:01.050412 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:01.050374    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ktm2t" event={"ID":"fc4d0910-9984-483e-9079-22b47c0defdb","Type":"ContainerStarted","Data":"1fe27bcc6e11db8502fc6e47b9960e56ce58db6e66abc8a8910575762901778c"}
Jul 07 17:08:01.069363 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:01.069266    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-j6qcn" event={"ID":"49ea513d-8038-4a64-923b-5a32649fa089","Type":"ContainerStarted","Data":"fd1ead3941d31b6c81f6f5ba08bad4c0f5702739da9f83777782911681dccad5"}
Jul 07 17:08:01.080447 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:01.080347    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vld2l" event={"ID":"66c074da-39ba-4e2e-ada6-9f71c051f3ec","Type":"ContainerStarted","Data":"126e122aa966753f852f71e4aa3504594dd29ce21ec5e12542ce5c935d6c9974"}
Jul 07 17:08:01.090177 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:01.090139    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz" event={"ID":"4a99fb8d-bf83-4929-8838-c1649ba58cfb","Type":"ContainerStarted","Data":"34641d56becceb277cd861f4126e9df5d5e95b6a3f8f489bb0b68eedf87478bf"}
Jul 07 17:08:01.099385 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:01.099277    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-66zv2" event={"ID":"ecf1ac63-1ea5-4590-8d60-b199e5eff959","Type":"ContainerStarted","Data":"ed736b7e94d66b61887b6a8e3283d93b9d6e98ac3fd6eecb63c90c78d7384a48"}
Jul 07 17:08:01.122518 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:01.122473    2579 generic.go:356] "Generic (PLEG): container finished" podID="c3ab2f1cccd249a4a2fa498a1527f9d2" containerID="f86984d6c3a9587034a32e82698ce2bef68112afb83ed5739bc831499e799009" exitCode=0
Jul 07 17:08:01.123053 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:01.123020    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-103.ec2.internal" event={"ID":"c3ab2f1cccd249a4a2fa498a1527f9d2","Type":"ContainerDied","Data":"f86984d6c3a9587034a32e82698ce2bef68112afb83ed5739bc831499e799009"}
Jul 07 17:08:01.133446 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:01.133312    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-87n7p" event={"ID":"187a8f3e-75c8-461a-a0f0-f3e3723d3a34","Type":"ContainerStarted","Data":"e1d0906c9accd3259d59979d3c1bad24b876ae247dbd6bcf008fb250efda2a6c"}
Jul 07 17:08:01.136500 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:01.136426    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4h5b6" event={"ID":"1c55e58a-1b4a-4643-923c-c235327231ed","Type":"ContainerStarted","Data":"dd235f3508a484de5806f9b3d92a8f70319154b51032c754b72f3051b1283d38"}
Jul 07 17:08:01.144893 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:01.144829    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2mpbm" event={"ID":"a4018c2b-fdcb-454a-b094-c2ed4e08e8ef","Type":"ContainerStarted","Data":"6120ed4d89a31ca3b907759fbc24577827d5d5d7f09d09f6b542148996beb4fb"}
Jul 07 17:08:01.155402 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:01.155358    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ls9b9" event={"ID":"5e44079c-1f38-4216-bb3b-9bee9ce4338d","Type":"ContainerStarted","Data":"02f1f02396bffd103f88c777f56a41d3dba248681777adb98e8510a9dda67d2f"}
Jul 07 17:08:01.163562 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:01.163502    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-103.ec2.internal" podStartSLOduration=2.163481518 podStartE2EDuration="2.163481518s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:07:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:08:01.079090478 +0000 UTC m=+4.575168991" watchObservedRunningTime="2026-07-07 17:08:01.163481518 +0000 UTC m=+4.659560031"
Jul 07 17:08:02.022445 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:02.022406    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fm4tg"
Jul 07 17:08:02.022974 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:02.022549    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fm4tg" podUID="d6844e3d-9b85-4d30-86db-35466ed9a36e"
Jul 07 17:08:02.187657 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:02.184023    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-103.ec2.internal" event={"ID":"c3ab2f1cccd249a4a2fa498a1527f9d2","Type":"ContainerStarted","Data":"0f66357403048de46f7ebb66b391912abb5dcafdcf7785e11650a3b12fd4eb50"}
Jul 07 17:08:02.210503 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:02.210439    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-103.ec2.internal" podStartSLOduration=3.210415764 podStartE2EDuration="3.210415764s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:07:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:08:02.209717948 +0000 UTC m=+5.705796461" watchObservedRunningTime="2026-07-07 17:08:02.210415764 +0000 UTC m=+5.706494280"
Jul 07 17:08:02.506587 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:02.506495    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c398fff8-ec63-4063-b3a4-92bf0aa5df9c-metrics-certs\") pod \"network-metrics-daemon-24rwk\" (UID: \"c398fff8-ec63-4063-b3a4-92bf0aa5df9c\") " pod="openshift-multus/network-metrics-daemon-24rwk"
Jul 07 17:08:02.506773 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:02.506678    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 17:08:02.506835 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:02.506780    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c398fff8-ec63-4063-b3a4-92bf0aa5df9c-metrics-certs podName:c398fff8-ec63-4063-b3a4-92bf0aa5df9c nodeName:}" failed. No retries permitted until 2026-07-07 17:08:06.506758142 +0000 UTC m=+10.002836637 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c398fff8-ec63-4063-b3a4-92bf0aa5df9c-metrics-certs") pod "network-metrics-daemon-24rwk" (UID: "c398fff8-ec63-4063-b3a4-92bf0aa5df9c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 17:08:02.607787 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:02.607746    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbv8k\" (UniqueName: \"kubernetes.io/projected/d6844e3d-9b85-4d30-86db-35466ed9a36e-kube-api-access-jbv8k\") pod \"network-check-target-fm4tg\" (UID: \"d6844e3d-9b85-4d30-86db-35466ed9a36e\") " pod="openshift-network-diagnostics/network-check-target-fm4tg"
Jul 07 17:08:02.608046 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:02.607804    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5d706921-ddd7-4b39-840d-a03cef437d64-original-pull-secret\") pod \"global-pull-secret-syncer-wqh28\" (UID: \"5d706921-ddd7-4b39-840d-a03cef437d64\") " pod="kube-system/global-pull-secret-syncer-wqh28"
Jul 07 17:08:02.608046 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:02.607982    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 17:08:02.608046 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:02.607990    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 17:08:02.608046 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:02.608010    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 17:08:02.608046 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:02.608024    2579 projected.go:196] Error preparing data for projected volume kube-api-access-jbv8k for pod openshift-network-diagnostics/network-check-target-fm4tg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 17:08:02.608325 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:02.608068    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d706921-ddd7-4b39-840d-a03cef437d64-original-pull-secret podName:5d706921-ddd7-4b39-840d-a03cef437d64 nodeName:}" failed. No retries permitted until 2026-07-07 17:08:06.608048132 +0000 UTC m=+10.104126635 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5d706921-ddd7-4b39-840d-a03cef437d64-original-pull-secret") pod "global-pull-secret-syncer-wqh28" (UID: "5d706921-ddd7-4b39-840d-a03cef437d64") : object "kube-system"/"original-pull-secret" not registered
Jul 07 17:08:02.608325 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:02.608091    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d6844e3d-9b85-4d30-86db-35466ed9a36e-kube-api-access-jbv8k podName:d6844e3d-9b85-4d30-86db-35466ed9a36e nodeName:}" failed. No retries permitted until 2026-07-07 17:08:06.608080382 +0000 UTC m=+10.104158874 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-jbv8k" (UniqueName: "kubernetes.io/projected/d6844e3d-9b85-4d30-86db-35466ed9a36e-kube-api-access-jbv8k") pod "network-check-target-fm4tg" (UID: "d6844e3d-9b85-4d30-86db-35466ed9a36e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 17:08:03.025624 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:03.025595    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqh28"
Jul 07 17:08:03.026127 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:03.025725    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqh28" podUID="5d706921-ddd7-4b39-840d-a03cef437d64"
Jul 07 17:08:03.026127 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:03.025793    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-24rwk"
Jul 07 17:08:03.026127 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:03.025982    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-24rwk" podUID="c398fff8-ec63-4063-b3a4-92bf0aa5df9c"
Jul 07 17:08:04.022105 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:04.022066    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fm4tg"
Jul 07 17:08:04.022316 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:04.022204    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fm4tg" podUID="d6844e3d-9b85-4d30-86db-35466ed9a36e"
Jul 07 17:08:05.026460 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:05.026364    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-24rwk"
Jul 07 17:08:05.026850 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:05.026512    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-24rwk" podUID="c398fff8-ec63-4063-b3a4-92bf0aa5df9c"
Jul 07 17:08:05.027114 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:05.026933    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqh28"
Jul 07 17:08:05.027114 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:05.027069    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqh28" podUID="5d706921-ddd7-4b39-840d-a03cef437d64"
Jul 07 17:08:06.022260 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:06.022223    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fm4tg"
Jul 07 17:08:06.022455 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:06.022366    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fm4tg" podUID="d6844e3d-9b85-4d30-86db-35466ed9a36e"
Jul 07 17:08:06.540179 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:06.540136    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c398fff8-ec63-4063-b3a4-92bf0aa5df9c-metrics-certs\") pod \"network-metrics-daemon-24rwk\" (UID: \"c398fff8-ec63-4063-b3a4-92bf0aa5df9c\") " pod="openshift-multus/network-metrics-daemon-24rwk"
Jul 07 17:08:06.540558 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:06.540297    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 17:08:06.540558 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:06.540386    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c398fff8-ec63-4063-b3a4-92bf0aa5df9c-metrics-certs podName:c398fff8-ec63-4063-b3a4-92bf0aa5df9c nodeName:}" failed. No retries permitted until 2026-07-07 17:08:14.540363763 +0000 UTC m=+18.036442259 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c398fff8-ec63-4063-b3a4-92bf0aa5df9c-metrics-certs") pod "network-metrics-daemon-24rwk" (UID: "c398fff8-ec63-4063-b3a4-92bf0aa5df9c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 17:08:06.640970 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:06.640928    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbv8k\" (UniqueName: \"kubernetes.io/projected/d6844e3d-9b85-4d30-86db-35466ed9a36e-kube-api-access-jbv8k\") pod \"network-check-target-fm4tg\" (UID: \"d6844e3d-9b85-4d30-86db-35466ed9a36e\") " pod="openshift-network-diagnostics/network-check-target-fm4tg"
Jul 07 17:08:06.641156 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:06.641001    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5d706921-ddd7-4b39-840d-a03cef437d64-original-pull-secret\") pod \"global-pull-secret-syncer-wqh28\" (UID: \"5d706921-ddd7-4b39-840d-a03cef437d64\") " pod="kube-system/global-pull-secret-syncer-wqh28"
Jul 07 17:08:06.641156 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:06.641118    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 17:08:06.641268 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:06.641180    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d706921-ddd7-4b39-840d-a03cef437d64-original-pull-secret podName:5d706921-ddd7-4b39-840d-a03cef437d64 nodeName:}" failed. No retries permitted until 2026-07-07 17:08:14.64116102 +0000 UTC m=+18.137239532 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5d706921-ddd7-4b39-840d-a03cef437d64-original-pull-secret") pod "global-pull-secret-syncer-wqh28" (UID: "5d706921-ddd7-4b39-840d-a03cef437d64") : object "kube-system"/"original-pull-secret" not registered
Jul 07 17:08:06.641710 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:06.641682    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 17:08:06.641710 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:06.641708    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 17:08:06.641892 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:06.641722    2579 projected.go:196] Error preparing data for projected volume kube-api-access-jbv8k for pod openshift-network-diagnostics/network-check-target-fm4tg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 17:08:06.641892 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:06.641775    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d6844e3d-9b85-4d30-86db-35466ed9a36e-kube-api-access-jbv8k podName:d6844e3d-9b85-4d30-86db-35466ed9a36e nodeName:}" failed. No retries permitted until 2026-07-07 17:08:14.641757406 +0000 UTC m=+18.137835908 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-jbv8k" (UniqueName: "kubernetes.io/projected/d6844e3d-9b85-4d30-86db-35466ed9a36e-kube-api-access-jbv8k") pod "network-check-target-fm4tg" (UID: "d6844e3d-9b85-4d30-86db-35466ed9a36e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 17:08:07.023032 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:07.022941    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqh28"
Jul 07 17:08:07.023191 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:07.023114    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqh28" podUID="5d706921-ddd7-4b39-840d-a03cef437d64"
Jul 07 17:08:07.023191 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:07.023128    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-24rwk"
Jul 07 17:08:07.023296 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:07.023258    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-24rwk" podUID="c398fff8-ec63-4063-b3a4-92bf0aa5df9c"
Jul 07 17:08:08.021939 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:08.021885    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fm4tg"
Jul 07 17:08:08.022427 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:08.022034    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fm4tg" podUID="d6844e3d-9b85-4d30-86db-35466ed9a36e"
Jul 07 17:08:09.022549 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:09.022511    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-24rwk"
Jul 07 17:08:09.023052 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:09.022680    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-24rwk" podUID="c398fff8-ec63-4063-b3a4-92bf0aa5df9c"
Jul 07 17:08:09.023176 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:09.023151    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqh28"
Jul 07 17:08:09.023612 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:09.023255    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqh28" podUID="5d706921-ddd7-4b39-840d-a03cef437d64"
Jul 07 17:08:10.022414 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:10.022379    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fm4tg"
Jul 07 17:08:10.022555 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:10.022481    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fm4tg" podUID="d6844e3d-9b85-4d30-86db-35466ed9a36e"
Jul 07 17:08:11.022347 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:11.022316    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqh28"
Jul 07 17:08:11.022761 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:11.022433    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqh28" podUID="5d706921-ddd7-4b39-840d-a03cef437d64"
Jul 07 17:08:11.022761 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:11.022489    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-24rwk"
Jul 07 17:08:11.022761 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:11.022613    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-24rwk" podUID="c398fff8-ec63-4063-b3a4-92bf0aa5df9c"
Jul 07 17:08:12.021938 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:12.021839    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fm4tg"
Jul 07 17:08:12.022102 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:12.021993    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fm4tg" podUID="d6844e3d-9b85-4d30-86db-35466ed9a36e"
Jul 07 17:08:13.022175 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:13.022137    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqh28"
Jul 07 17:08:13.022635 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:13.022275    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqh28" podUID="5d706921-ddd7-4b39-840d-a03cef437d64"
Jul 07 17:08:13.022635 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:13.022330    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-24rwk"
Jul 07 17:08:13.022635 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:13.022465    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-24rwk" podUID="c398fff8-ec63-4063-b3a4-92bf0aa5df9c"
Jul 07 17:08:14.021958 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:14.021922    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fm4tg"
Jul 07 17:08:14.022163 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:14.022048    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fm4tg" podUID="d6844e3d-9b85-4d30-86db-35466ed9a36e"
Jul 07 17:08:14.603709 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:14.603679    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c398fff8-ec63-4063-b3a4-92bf0aa5df9c-metrics-certs\") pod \"network-metrics-daemon-24rwk\" (UID: \"c398fff8-ec63-4063-b3a4-92bf0aa5df9c\") " pod="openshift-multus/network-metrics-daemon-24rwk"
Jul 07 17:08:14.604147 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:14.603791    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 17:08:14.604147 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:14.603850    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c398fff8-ec63-4063-b3a4-92bf0aa5df9c-metrics-certs podName:c398fff8-ec63-4063-b3a4-92bf0aa5df9c nodeName:}" failed. No retries permitted until 2026-07-07 17:08:30.603835796 +0000 UTC m=+34.099914295 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c398fff8-ec63-4063-b3a4-92bf0aa5df9c-metrics-certs") pod "network-metrics-daemon-24rwk" (UID: "c398fff8-ec63-4063-b3a4-92bf0aa5df9c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 17:08:14.704282 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:14.704240    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbv8k\" (UniqueName: \"kubernetes.io/projected/d6844e3d-9b85-4d30-86db-35466ed9a36e-kube-api-access-jbv8k\") pod \"network-check-target-fm4tg\" (UID: \"d6844e3d-9b85-4d30-86db-35466ed9a36e\") " pod="openshift-network-diagnostics/network-check-target-fm4tg"
Jul 07 17:08:14.704282 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:14.704288    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5d706921-ddd7-4b39-840d-a03cef437d64-original-pull-secret\") pod \"global-pull-secret-syncer-wqh28\" (UID: \"5d706921-ddd7-4b39-840d-a03cef437d64\") " pod="kube-system/global-pull-secret-syncer-wqh28"
Jul 07 17:08:14.704501 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:14.704406    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 17:08:14.704501 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:14.704434    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 17:08:14.704501 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:14.704458    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 17:08:14.704501 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:14.704466    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d706921-ddd7-4b39-840d-a03cef437d64-original-pull-secret podName:5d706921-ddd7-4b39-840d-a03cef437d64 nodeName:}" failed. No retries permitted until 2026-07-07 17:08:30.704451778 +0000 UTC m=+34.200530266 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5d706921-ddd7-4b39-840d-a03cef437d64-original-pull-secret") pod "global-pull-secret-syncer-wqh28" (UID: "5d706921-ddd7-4b39-840d-a03cef437d64") : object "kube-system"/"original-pull-secret" not registered
Jul 07 17:08:14.704501 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:14.704470    2579 projected.go:196] Error preparing data for projected volume kube-api-access-jbv8k for pod openshift-network-diagnostics/network-check-target-fm4tg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 17:08:14.704694 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:14.704515    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d6844e3d-9b85-4d30-86db-35466ed9a36e-kube-api-access-jbv8k podName:d6844e3d-9b85-4d30-86db-35466ed9a36e nodeName:}" failed. No retries permitted until 2026-07-07 17:08:30.704500402 +0000 UTC m=+34.200578901 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-jbv8k" (UniqueName: "kubernetes.io/projected/d6844e3d-9b85-4d30-86db-35466ed9a36e-kube-api-access-jbv8k") pod "network-check-target-fm4tg" (UID: "d6844e3d-9b85-4d30-86db-35466ed9a36e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 17:08:15.021842 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:15.021761    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqh28"
Jul 07 17:08:15.021842 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:15.021796    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-24rwk"
Jul 07 17:08:15.022061 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:15.021921    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqh28" podUID="5d706921-ddd7-4b39-840d-a03cef437d64"
Jul 07 17:08:15.022169 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:15.022058    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-24rwk" podUID="c398fff8-ec63-4063-b3a4-92bf0aa5df9c"
Jul 07 17:08:16.021597 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:16.021562    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fm4tg"
Jul 07 17:08:16.022043 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:16.021688    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fm4tg" podUID="d6844e3d-9b85-4d30-86db-35466ed9a36e"
Jul 07 17:08:17.022406 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:17.022370    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqh28"
Jul 07 17:08:17.022777 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:17.022473    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqh28" podUID="5d706921-ddd7-4b39-840d-a03cef437d64"
Jul 07 17:08:17.022777 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:17.022575    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-24rwk"
Jul 07 17:08:17.022777 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:17.022704    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-24rwk" podUID="c398fff8-ec63-4063-b3a4-92bf0aa5df9c"
Jul 07 17:08:18.022421 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:18.022389    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fm4tg"
Jul 07 17:08:18.022832 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:18.022493    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fm4tg" podUID="d6844e3d-9b85-4d30-86db-35466ed9a36e"
Jul 07 17:08:19.022687 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:19.022233    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-24rwk"
Jul 07 17:08:19.022687 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:19.022364    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-24rwk" podUID="c398fff8-ec63-4063-b3a4-92bf0aa5df9c"
Jul 07 17:08:19.022687 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:19.022423    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqh28"
Jul 07 17:08:19.022687 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:19.022512    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqh28" podUID="5d706921-ddd7-4b39-840d-a03cef437d64"
Jul 07 17:08:20.021787 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:20.021426    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fm4tg"
Jul 07 17:08:20.021926 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:20.021883    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fm4tg" podUID="d6844e3d-9b85-4d30-86db-35466ed9a36e"
Jul 07 17:08:20.227155 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:20.227125    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ls9b9" event={"ID":"5e44079c-1f38-4216-bb3b-9bee9ce4338d","Type":"ContainerStarted","Data":"4fca57d232516b430bc7820bfcfe27d80afa55e71c9abd3d1914eac21805c898"}
Jul 07 17:08:20.228727 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:20.228706    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ktm2t" event={"ID":"fc4d0910-9984-483e-9079-22b47c0defdb","Type":"ContainerStarted","Data":"da68d9a53f2c013a413980a03a7db4b602ebf317a00b6c7c8255d8c84b74b4ca"}
Jul 07 17:08:20.230202 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:20.230169    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vld2l" event={"ID":"66c074da-39ba-4e2e-ada6-9f71c051f3ec","Type":"ContainerStarted","Data":"a70ef8f9958404c646ddaba584b774b06d991c1c1871678e7104875b59a4c747"}
Jul 07 17:08:20.232781 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:20.232750    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz" event={"ID":"4a99fb8d-bf83-4929-8838-c1649ba58cfb","Type":"ContainerStarted","Data":"76b97d5159e62f7e229132218ef9113efd4d7925642e05d55037cc20737df018"}
Jul 07 17:08:20.232869 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:20.232812    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz" event={"ID":"4a99fb8d-bf83-4929-8838-c1649ba58cfb","Type":"ContainerStarted","Data":"f052b7a5394524487b01c96cf20a21a88d19f273dde745bd53bcff86e213a083"}
Jul 07 17:08:20.232869 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:20.232843    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz" event={"ID":"4a99fb8d-bf83-4929-8838-c1649ba58cfb","Type":"ContainerStarted","Data":"619dc6fe3cef779404e6e87eeab18045913b3f825c91851c4f1d6c19903ed826"}
Jul 07 17:08:20.232976 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:20.232879    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz" event={"ID":"4a99fb8d-bf83-4929-8838-c1649ba58cfb","Type":"ContainerStarted","Data":"344cf861dc92b0b174df3f3629c5c08a0d1641c970d34df34e9e87f438f7144c"}
Jul 07 17:08:20.234000 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:20.233977    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-66zv2" event={"ID":"ecf1ac63-1ea5-4590-8d60-b199e5eff959","Type":"ContainerStarted","Data":"472818e92948e4694ac9bfbb5127739c9c17754c2652db3a3be537ef781f0217"}
Jul 07 17:08:20.235178 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:20.235159    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-87n7p" event={"ID":"187a8f3e-75c8-461a-a0f0-f3e3723d3a34","Type":"ContainerStarted","Data":"4ae72ea2eadfc5efc0b58d0cef3b359e62916ec31575a023421b164f40eb1b2b"}
Jul 07 17:08:20.236365 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:20.236343    2579 generic.go:356] "Generic (PLEG): container finished" podID="1c55e58a-1b4a-4643-923c-c235327231ed" containerID="0b1a8b99f050807923a61125882cbc0dd81e2944e367909e46fcc2f62eeba4b5" exitCode=0
Jul 07 17:08:20.236452 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:20.236418    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4h5b6" event={"ID":"1c55e58a-1b4a-4643-923c-c235327231ed","Type":"ContainerDied","Data":"0b1a8b99f050807923a61125882cbc0dd81e2944e367909e46fcc2f62eeba4b5"}
Jul 07 17:08:20.237710 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:20.237684    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2mpbm" event={"ID":"a4018c2b-fdcb-454a-b094-c2ed4e08e8ef","Type":"ContainerStarted","Data":"5bb5cf76413a837fe93cfae90d7c71078ac343054bedd7124a30ef428dea0493"}
Jul 07 17:08:20.252345 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:20.252312    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-ls9b9" podStartSLOduration=2.592161333 podStartE2EDuration="22.252301159s" totalImagesPullingTime="19.660139826s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:07:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:08:20.251841646 +0000 UTC m=+23.747920153" watchObservedRunningTime="2026-07-07 17:08:20.252301159 +0000 UTC m=+23.748379667"
Jul 07 17:08:20.279045 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:20.279013    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-ktm2t" podStartSLOduration=6.887062507 podStartE2EDuration="22.279001594s" totalImagesPullingTime="15.391939087s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:07:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:08:20.278515091 +0000 UTC m=+23.774593651" watchObservedRunningTime="2026-07-07 17:08:20.279001594 +0000 UTC m=+23.775080101"
Jul 07 17:08:20.296280 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:20.296241    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-2mpbm" podStartSLOduration=2.701052264 podStartE2EDuration="22.296227087s" totalImagesPullingTime="19.595174823s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:07:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:08:20.29622741 +0000 UTC m=+23.792305922" watchObservedRunningTime="2026-07-07 17:08:20.296227087 +0000 UTC m=+23.792305595"
Jul 07 17:08:20.334949 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:20.334880    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-66zv2" podStartSLOduration=2.735969612 podStartE2EDuration="22.334861451s" totalImagesPullingTime="19.598891839s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:07:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:08:20.314273868 +0000 UTC m=+23.810352377" watchObservedRunningTime="2026-07-07 17:08:20.334861451 +0000 UTC m=+23.830939960"
Jul 07 17:08:20.335042 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:20.334995    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-87n7p" podStartSLOduration=2.747886061 podStartE2EDuration="22.334988484s" totalImagesPullingTime="19.587102423s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:07:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:08:20.334450814 +0000 UTC m=+23.830529323" watchObservedRunningTime="2026-07-07 17:08:20.334988484 +0000 UTC m=+23.831066994"
Jul 07 17:08:21.021479 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:21.021452    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqh28"
Jul 07 17:08:21.021600 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:21.021565    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqh28" podUID="5d706921-ddd7-4b39-840d-a03cef437d64"
Jul 07 17:08:21.021661 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:21.021640    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-24rwk"
Jul 07 17:08:21.021741 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:21.021722    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-24rwk" podUID="c398fff8-ec63-4063-b3a4-92bf0aa5df9c"
Jul 07 17:08:21.104079 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:21.104051    2579 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 07 17:08:21.241860 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:21.241772    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vld2l" event={"ID":"66c074da-39ba-4e2e-ada6-9f71c051f3ec","Type":"ContainerStarted","Data":"6cb767f6d1f7e11d0341d3a41662d6823e625ce86206f769cee0023d2e9b3175"}
Jul 07 17:08:21.245222 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:21.245187    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz" event={"ID":"4a99fb8d-bf83-4929-8838-c1649ba58cfb","Type":"ContainerStarted","Data":"7dbca9664ec540a985c4fb80c04a360a74825057a9d8ab4f25fd8b17f13cead7"}
Jul 07 17:08:21.245367 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:21.245258    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz" event={"ID":"4a99fb8d-bf83-4929-8838-c1649ba58cfb","Type":"ContainerStarted","Data":"6dd78b7c1da5927015d9b9aeb2702ef1c30084c1bf858f39d1b85a156db4936d"}
Jul 07 17:08:21.246731 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:21.246702    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-j6qcn" event={"ID":"49ea513d-8038-4a64-923b-5a32649fa089","Type":"ContainerStarted","Data":"c8e1d4080dd2d93faa110cf5bd3985efc367a69b0a090240e1dab5196d882d96"}
Jul 07 17:08:21.264625 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:21.264579    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-j6qcn" podStartSLOduration=3.67725051 podStartE2EDuration="23.264561339s" totalImagesPullingTime="19.587310829s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:07:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:08:21.264188162 +0000 UTC m=+24.760266669" watchObservedRunningTime="2026-07-07 17:08:21.264561339 +0000 UTC m=+24.760639848"
Jul 07 17:08:21.942365 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:21.942259    2579 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T17:08:21.104073345Z","UUID":"6ae6ede1-57fa-4a98-941c-edf997952b7d","Handler":null,"Name":"","Endpoint":""}
Jul 07 17:08:21.943713 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:21.943686    2579 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 07 17:08:21.943853 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:21.943729    2579 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 07 17:08:22.022212 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:22.022183    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fm4tg"
Jul 07 17:08:22.022363 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:22.022289    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fm4tg" podUID="d6844e3d-9b85-4d30-86db-35466ed9a36e"
Jul 07 17:08:22.252121 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:22.252039    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vld2l" event={"ID":"66c074da-39ba-4e2e-ada6-9f71c051f3ec","Type":"ContainerStarted","Data":"605b4e3786dce99e95b3384725295ede78154c9d6f24bec399c1d3062ca01b0c"}
Jul 07 17:08:23.022374 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:23.022336    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqh28"
Jul 07 17:08:23.022552 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:23.022345    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-24rwk"
Jul 07 17:08:23.022552 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:23.022474    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqh28" podUID="5d706921-ddd7-4b39-840d-a03cef437d64"
Jul 07 17:08:23.022552 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:23.022534    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-24rwk" podUID="c398fff8-ec63-4063-b3a4-92bf0aa5df9c"
Jul 07 17:08:23.257956 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:23.257906    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz" event={"ID":"4a99fb8d-bf83-4929-8838-c1649ba58cfb","Type":"ContainerStarted","Data":"f4597586498e0a69e78eaba9cdafe8e6706f6277fc9a97a4828cb378feb7fc49"}
Jul 07 17:08:24.022173 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:24.022137    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fm4tg"
Jul 07 17:08:24.022315 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:24.022247    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fm4tg" podUID="d6844e3d-9b85-4d30-86db-35466ed9a36e"
Jul 07 17:08:24.233203 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:24.233173    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-ktm2t"
Jul 07 17:08:24.233747 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:24.233722    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-ktm2t"
Jul 07 17:08:24.253655 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:24.253612    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vld2l" podStartSLOduration=4.638274097 podStartE2EDuration="26.253595362s" totalImagesPullingTime="21.615321265s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:07:58 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:08:22.278923484 +0000 UTC m=+25.775001993" watchObservedRunningTime="2026-07-07 17:08:24.253595362 +0000 UTC m=+27.749673874"
Jul 07 17:08:24.259937 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:24.259865    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-ktm2t"
Jul 07 17:08:24.260521 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:24.260339    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-ktm2t"
Jul 07 17:08:25.022471 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:25.022390    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqh28"
Jul 07 17:08:25.022628 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:25.022397    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-24rwk"
Jul 07 17:08:25.022628 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:25.022512    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqh28" podUID="5d706921-ddd7-4b39-840d-a03cef437d64"
Jul 07 17:08:25.022628 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:25.022618    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-24rwk" podUID="c398fff8-ec63-4063-b3a4-92bf0aa5df9c"
Jul 07 17:08:26.021968 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:26.021768    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fm4tg"
Jul 07 17:08:26.022425 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:26.022005    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fm4tg" podUID="d6844e3d-9b85-4d30-86db-35466ed9a36e"
Jul 07 17:08:26.267826 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:26.267791    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz" event={"ID":"4a99fb8d-bf83-4929-8838-c1649ba58cfb","Type":"ContainerStarted","Data":"1740f0af7e3ecdd50751725ea5b5be0bee2459203a354d82c60b82928cb89137"}
Jul 07 17:08:26.268093 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:26.268072    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:08:26.269571 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:26.269547    2579 generic.go:356] "Generic (PLEG): container finished" podID="1c55e58a-1b4a-4643-923c-c235327231ed" containerID="a94e17e5955d8b834cb20c08145b040883a587b89a73f0dedb60f2274e7af1ef" exitCode=0
Jul 07 17:08:26.269667 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:26.269634    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4h5b6" event={"ID":"1c55e58a-1b4a-4643-923c-c235327231ed","Type":"ContainerDied","Data":"a94e17e5955d8b834cb20c08145b040883a587b89a73f0dedb60f2274e7af1ef"}
Jul 07 17:08:26.284165 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:26.284147    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:08:26.298433 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:26.298394    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz" podStartSLOduration=8.640006843 podStartE2EDuration="28.298381634s" totalImagesPullingTime="19.658374791s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:07:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:08:26.297262495 +0000 UTC m=+29.793341003" watchObservedRunningTime="2026-07-07 17:08:26.298381634 +0000 UTC m=+29.794460141"
Jul 07 17:08:27.023279 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:27.023230    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqh28"
Jul 07 17:08:27.023807 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:27.023352    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqh28" podUID="5d706921-ddd7-4b39-840d-a03cef437d64"
Jul 07 17:08:27.023873 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:27.023848    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-24rwk"
Jul 07 17:08:27.024022 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:27.023998    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-24rwk" podUID="c398fff8-ec63-4063-b3a4-92bf0aa5df9c"
Jul 07 17:08:27.272408 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:27.272378    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:08:27.272549 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:27.272473    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:08:27.288911 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:27.288829    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:08:27.523436 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:27.523286    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fm4tg"]
Jul 07 17:08:27.523529 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:27.523516    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fm4tg"
Jul 07 17:08:27.523614 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:27.523599    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fm4tg" podUID="d6844e3d-9b85-4d30-86db-35466ed9a36e"
Jul 07 17:08:27.526632 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:27.526611    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wqh28"]
Jul 07 17:08:27.526744 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:27.526696    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqh28"
Jul 07 17:08:27.526817 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:27.526795    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqh28" podUID="5d706921-ddd7-4b39-840d-a03cef437d64"
Jul 07 17:08:27.527466 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:27.527434    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-24rwk"]
Jul 07 17:08:27.527548 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:27.527531    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-24rwk"
Jul 07 17:08:27.527660 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:27.527638    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-24rwk" podUID="c398fff8-ec63-4063-b3a4-92bf0aa5df9c"
Jul 07 17:08:28.275983 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:28.275951    2579 generic.go:356] "Generic (PLEG): container finished" podID="1c55e58a-1b4a-4643-923c-c235327231ed" containerID="ae2cf1ed06cd25084b147ddf15129a5d024f21efe81687ef0082729ba32d59ac" exitCode=0
Jul 07 17:08:28.276408 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:28.276044    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4h5b6" event={"ID":"1c55e58a-1b4a-4643-923c-c235327231ed","Type":"ContainerDied","Data":"ae2cf1ed06cd25084b147ddf15129a5d024f21efe81687ef0082729ba32d59ac"}
Jul 07 17:08:29.022483 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:29.022395    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fm4tg"
Jul 07 17:08:29.022483 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:29.022408    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-24rwk"
Jul 07 17:08:29.022678 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:29.022521    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fm4tg" podUID="d6844e3d-9b85-4d30-86db-35466ed9a36e"
Jul 07 17:08:29.022678 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:29.022408    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqh28"
Jul 07 17:08:29.022678 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:29.022579    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-24rwk" podUID="c398fff8-ec63-4063-b3a4-92bf0aa5df9c"
Jul 07 17:08:29.022812 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:29.022682    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqh28" podUID="5d706921-ddd7-4b39-840d-a03cef437d64"
Jul 07 17:08:30.283258 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:30.283227    2579 generic.go:356] "Generic (PLEG): container finished" podID="1c55e58a-1b4a-4643-923c-c235327231ed" containerID="be6161c1dbfbad47ce9362cdd371566e2f12fa31ed835377b6de98aec796b98c" exitCode=0
Jul 07 17:08:30.283811 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:30.283267    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4h5b6" event={"ID":"1c55e58a-1b4a-4643-923c-c235327231ed","Type":"ContainerDied","Data":"be6161c1dbfbad47ce9362cdd371566e2f12fa31ed835377b6de98aec796b98c"}
Jul 07 17:08:30.626237 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:30.626175    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c398fff8-ec63-4063-b3a4-92bf0aa5df9c-metrics-certs\") pod \"network-metrics-daemon-24rwk\" (UID: \"c398fff8-ec63-4063-b3a4-92bf0aa5df9c\") " pod="openshift-multus/network-metrics-daemon-24rwk"
Jul 07 17:08:30.626348 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:30.626261    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 17:08:30.626348 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:30.626312    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c398fff8-ec63-4063-b3a4-92bf0aa5df9c-metrics-certs podName:c398fff8-ec63-4063-b3a4-92bf0aa5df9c nodeName:}" failed. No retries permitted until 2026-07-07 17:09:02.626292031 +0000 UTC m=+66.122370530 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c398fff8-ec63-4063-b3a4-92bf0aa5df9c-metrics-certs") pod "network-metrics-daemon-24rwk" (UID: "c398fff8-ec63-4063-b3a4-92bf0aa5df9c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 17:08:30.726495 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:30.726472    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbv8k\" (UniqueName: \"kubernetes.io/projected/d6844e3d-9b85-4d30-86db-35466ed9a36e-kube-api-access-jbv8k\") pod \"network-check-target-fm4tg\" (UID: \"d6844e3d-9b85-4d30-86db-35466ed9a36e\") " pod="openshift-network-diagnostics/network-check-target-fm4tg"
Jul 07 17:08:30.726495 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:30.726505    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5d706921-ddd7-4b39-840d-a03cef437d64-original-pull-secret\") pod \"global-pull-secret-syncer-wqh28\" (UID: \"5d706921-ddd7-4b39-840d-a03cef437d64\") " pod="kube-system/global-pull-secret-syncer-wqh28"
Jul 07 17:08:30.726648 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:30.726607    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 17:08:30.726648 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:30.726632    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 17:08:30.726726 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:30.726656    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 17:08:30.726726 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:30.726666    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d706921-ddd7-4b39-840d-a03cef437d64-original-pull-secret podName:5d706921-ddd7-4b39-840d-a03cef437d64 nodeName:}" failed. No retries permitted until 2026-07-07 17:09:02.726652397 +0000 UTC m=+66.222730887 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5d706921-ddd7-4b39-840d-a03cef437d64-original-pull-secret") pod "global-pull-secret-syncer-wqh28" (UID: "5d706921-ddd7-4b39-840d-a03cef437d64") : object "kube-system"/"original-pull-secret" not registered
Jul 07 17:08:30.726726 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:30.726666    2579 projected.go:196] Error preparing data for projected volume kube-api-access-jbv8k for pod openshift-network-diagnostics/network-check-target-fm4tg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 17:08:30.726726 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:30.726710    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d6844e3d-9b85-4d30-86db-35466ed9a36e-kube-api-access-jbv8k podName:d6844e3d-9b85-4d30-86db-35466ed9a36e nodeName:}" failed. No retries permitted until 2026-07-07 17:09:02.726696184 +0000 UTC m=+66.222774674 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-jbv8k" (UniqueName: "kubernetes.io/projected/d6844e3d-9b85-4d30-86db-35466ed9a36e-kube-api-access-jbv8k") pod "network-check-target-fm4tg" (UID: "d6844e3d-9b85-4d30-86db-35466ed9a36e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 17:08:31.022413 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:31.022348    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-24rwk"
Jul 07 17:08:31.022534 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:31.022348    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fm4tg"
Jul 07 17:08:31.022534 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:31.022451    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-24rwk" podUID="c398fff8-ec63-4063-b3a4-92bf0aa5df9c"
Jul 07 17:08:31.022534 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:31.022348    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqh28"
Jul 07 17:08:31.022534 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:31.022515    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fm4tg" podUID="d6844e3d-9b85-4d30-86db-35466ed9a36e"
Jul 07 17:08:31.022679 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:31.022578    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqh28" podUID="5d706921-ddd7-4b39-840d-a03cef437d64"
Jul 07 17:08:32.275372 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:32.275352    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-103.ec2.internal" event="NodeReady"
Jul 07 17:08:32.275806 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:32.275451    2579 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 07 17:08:32.329952 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:32.329928    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-nbkp6"]
Jul 07 17:08:32.334137 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:32.334118    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nbkp6"
Jul 07 17:08:32.335581 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:32.335559    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-h7sqt"]
Jul 07 17:08:32.336394 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:32.336373    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 07 17:08:32.336594 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:32.336574    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9r2xr\""
Jul 07 17:08:32.336789 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:32.336768    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 07 17:08:32.339083 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:32.339065    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-h7sqt"
Jul 07 17:08:32.340877 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:32.340858    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 07 17:08:32.341226 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:32.341207    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 07 17:08:32.341323 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:32.341297    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-xf5p6\""
Jul 07 17:08:32.341668 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:32.341654    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 07 17:08:32.345380 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:32.345360    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nbkp6"]
Jul 07 17:08:32.351206 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:32.351185    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-h7sqt"]
Jul 07 17:08:32.441543 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:32.441516    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrqw9\" (UniqueName: \"kubernetes.io/projected/72d06848-e504-4952-953f-02b4800ddf45-kube-api-access-jrqw9\") pod \"ingress-canary-h7sqt\" (UID: \"72d06848-e504-4952-953f-02b4800ddf45\") " pod="openshift-ingress-canary/ingress-canary-h7sqt"
Jul 07 17:08:32.441665 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:32.441548    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsxpb\" (UniqueName: \"kubernetes.io/projected/b04e3abe-c8da-4123-ae78-3d978812545d-kube-api-access-fsxpb\") pod \"dns-default-nbkp6\" (UID: \"b04e3abe-c8da-4123-ae78-3d978812545d\") " pod="openshift-dns/dns-default-nbkp6"
Jul 07 17:08:32.441665 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:32.441573    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/72d06848-e504-4952-953f-02b4800ddf45-cert\") pod \"ingress-canary-h7sqt\" (UID: \"72d06848-e504-4952-953f-02b4800ddf45\") " pod="openshift-ingress-canary/ingress-canary-h7sqt"
Jul 07 17:08:32.441665 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:32.441643    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b04e3abe-c8da-4123-ae78-3d978812545d-config-volume\") pod \"dns-default-nbkp6\" (UID: \"b04e3abe-c8da-4123-ae78-3d978812545d\") " pod="openshift-dns/dns-default-nbkp6"
Jul 07 17:08:32.441779 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:32.441679    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b04e3abe-c8da-4123-ae78-3d978812545d-metrics-tls\") pod \"dns-default-nbkp6\" (UID: \"b04e3abe-c8da-4123-ae78-3d978812545d\") " pod="openshift-dns/dns-default-nbkp6"
Jul 07 17:08:32.441779 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:32.441701    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b04e3abe-c8da-4123-ae78-3d978812545d-tmp-dir\") pod \"dns-default-nbkp6\" (UID: \"b04e3abe-c8da-4123-ae78-3d978812545d\") " pod="openshift-dns/dns-default-nbkp6"
Jul 07 17:08:32.542869 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:32.542810    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrqw9\" (UniqueName: \"kubernetes.io/projected/72d06848-e504-4952-953f-02b4800ddf45-kube-api-access-jrqw9\") pod \"ingress-canary-h7sqt\" (UID: \"72d06848-e504-4952-953f-02b4800ddf45\") " pod="openshift-ingress-canary/ingress-canary-h7sqt"
Jul 07 17:08:32.542869 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:32.542841    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsxpb\" (UniqueName: \"kubernetes.io/projected/b04e3abe-c8da-4123-ae78-3d978812545d-kube-api-access-fsxpb\") pod \"dns-default-nbkp6\" (UID: \"b04e3abe-c8da-4123-ae78-3d978812545d\") " pod="openshift-dns/dns-default-nbkp6"
Jul 07 17:08:32.542869 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:32.542861    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/72d06848-e504-4952-953f-02b4800ddf45-cert\") pod \"ingress-canary-h7sqt\" (UID: \"72d06848-e504-4952-953f-02b4800ddf45\") " pod="openshift-ingress-canary/ingress-canary-h7sqt"
Jul 07 17:08:32.543072 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:32.542970    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b04e3abe-c8da-4123-ae78-3d978812545d-config-volume\") pod \"dns-default-nbkp6\" (UID: \"b04e3abe-c8da-4123-ae78-3d978812545d\") " pod="openshift-dns/dns-default-nbkp6"
Jul 07 17:08:32.543072 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:32.543036    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b04e3abe-c8da-4123-ae78-3d978812545d-metrics-tls\") pod \"dns-default-nbkp6\" (UID: \"b04e3abe-c8da-4123-ae78-3d978812545d\") " pod="openshift-dns/dns-default-nbkp6"
Jul 07 17:08:32.543072 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:32.543067    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b04e3abe-c8da-4123-ae78-3d978812545d-tmp-dir\") pod \"dns-default-nbkp6\" (UID: \"b04e3abe-c8da-4123-ae78-3d978812545d\") " pod="openshift-dns/dns-default-nbkp6"
Jul 07 17:08:32.543172 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:32.543149    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 17:08:32.543241 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:32.543228    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b04e3abe-c8da-4123-ae78-3d978812545d-metrics-tls podName:b04e3abe-c8da-4123-ae78-3d978812545d nodeName:}" failed. No retries permitted until 2026-07-07 17:08:33.043207439 +0000 UTC m=+36.539285931 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b04e3abe-c8da-4123-ae78-3d978812545d-metrics-tls") pod "dns-default-nbkp6" (UID: "b04e3abe-c8da-4123-ae78-3d978812545d") : secret "dns-default-metrics-tls" not found
Jul 07 17:08:32.543416 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:32.543392    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b04e3abe-c8da-4123-ae78-3d978812545d-tmp-dir\") pod \"dns-default-nbkp6\" (UID: \"b04e3abe-c8da-4123-ae78-3d978812545d\") " pod="openshift-dns/dns-default-nbkp6"
Jul 07 17:08:32.543570 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:32.543552    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b04e3abe-c8da-4123-ae78-3d978812545d-config-volume\") pod \"dns-default-nbkp6\" (UID: \"b04e3abe-c8da-4123-ae78-3d978812545d\") " pod="openshift-dns/dns-default-nbkp6"
Jul 07 17:08:32.546902 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:32.546860    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/72d06848-e504-4952-953f-02b4800ddf45-cert\") pod \"ingress-canary-h7sqt\" (UID: \"72d06848-e504-4952-953f-02b4800ddf45\") " pod="openshift-ingress-canary/ingress-canary-h7sqt"
Jul 07 17:08:32.552321 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:32.552295    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsxpb\" (UniqueName: \"kubernetes.io/projected/b04e3abe-c8da-4123-ae78-3d978812545d-kube-api-access-fsxpb\") pod \"dns-default-nbkp6\" (UID: \"b04e3abe-c8da-4123-ae78-3d978812545d\") " pod="openshift-dns/dns-default-nbkp6"
Jul 07 17:08:32.552527 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:32.552511    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrqw9\" (UniqueName: \"kubernetes.io/projected/72d06848-e504-4952-953f-02b4800ddf45-kube-api-access-jrqw9\") pod \"ingress-canary-h7sqt\" (UID: \"72d06848-e504-4952-953f-02b4800ddf45\") " pod="openshift-ingress-canary/ingress-canary-h7sqt"
Jul 07 17:08:32.654234 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:32.654209    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-h7sqt"
Jul 07 17:08:32.788141 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:32.787862    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-h7sqt"]
Jul 07 17:08:32.791025 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:08:32.790997    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72d06848_e504_4952_953f_02b4800ddf45.slice/crio-cd95031c119f6ebd157b9a3e31bc315b9977f46e5dbf1d236481782ab0dc9946 WatchSource:0}: Error finding container cd95031c119f6ebd157b9a3e31bc315b9977f46e5dbf1d236481782ab0dc9946: Status 404 returned error can't find the container with id cd95031c119f6ebd157b9a3e31bc315b9977f46e5dbf1d236481782ab0dc9946
Jul 07 17:08:33.021690 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:33.021662    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-24rwk"
Jul 07 17:08:33.021831 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:33.021662    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqh28"
Jul 07 17:08:33.021917 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:33.021662    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fm4tg"
Jul 07 17:08:33.024778 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:33.024758    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 07 17:08:33.024882 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:33.024872    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 07 17:08:33.024962 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:33.024948    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 07 17:08:33.025037 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:33.025020    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-chtgz\""
Jul 07 17:08:33.025436 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:33.025419    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jk68m\""
Jul 07 17:08:33.025524 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:33.025463    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 07 17:08:33.045574 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:33.045555    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b04e3abe-c8da-4123-ae78-3d978812545d-metrics-tls\") pod \"dns-default-nbkp6\" (UID: \"b04e3abe-c8da-4123-ae78-3d978812545d\") " pod="openshift-dns/dns-default-nbkp6"
Jul 07 17:08:33.045677 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:33.045663    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 17:08:33.045743 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:33.045726    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b04e3abe-c8da-4123-ae78-3d978812545d-metrics-tls podName:b04e3abe-c8da-4123-ae78-3d978812545d nodeName:}" failed. No retries permitted until 2026-07-07 17:08:34.045706982 +0000 UTC m=+37.541785484 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b04e3abe-c8da-4123-ae78-3d978812545d-metrics-tls") pod "dns-default-nbkp6" (UID: "b04e3abe-c8da-4123-ae78-3d978812545d") : secret "dns-default-metrics-tls" not found
Jul 07 17:08:33.293294 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:33.293265    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-h7sqt" event={"ID":"72d06848-e504-4952-953f-02b4800ddf45","Type":"ContainerStarted","Data":"cd95031c119f6ebd157b9a3e31bc315b9977f46e5dbf1d236481782ab0dc9946"}
Jul 07 17:08:34.053963 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:34.053931    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b04e3abe-c8da-4123-ae78-3d978812545d-metrics-tls\") pod \"dns-default-nbkp6\" (UID: \"b04e3abe-c8da-4123-ae78-3d978812545d\") " pod="openshift-dns/dns-default-nbkp6"
Jul 07 17:08:34.054139 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:34.054050    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 17:08:34.054139 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:34.054115    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b04e3abe-c8da-4123-ae78-3d978812545d-metrics-tls podName:b04e3abe-c8da-4123-ae78-3d978812545d nodeName:}" failed. No retries permitted until 2026-07-07 17:08:36.054097027 +0000 UTC m=+39.550175516 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b04e3abe-c8da-4123-ae78-3d978812545d-metrics-tls") pod "dns-default-nbkp6" (UID: "b04e3abe-c8da-4123-ae78-3d978812545d") : secret "dns-default-metrics-tls" not found
Jul 07 17:08:36.068422 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:36.068385    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b04e3abe-c8da-4123-ae78-3d978812545d-metrics-tls\") pod \"dns-default-nbkp6\" (UID: \"b04e3abe-c8da-4123-ae78-3d978812545d\") " pod="openshift-dns/dns-default-nbkp6"
Jul 07 17:08:36.069098 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:36.068537    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 17:08:36.069098 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:36.068592    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b04e3abe-c8da-4123-ae78-3d978812545d-metrics-tls podName:b04e3abe-c8da-4123-ae78-3d978812545d nodeName:}" failed. No retries permitted until 2026-07-07 17:08:40.068577681 +0000 UTC m=+43.564656185 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b04e3abe-c8da-4123-ae78-3d978812545d-metrics-tls") pod "dns-default-nbkp6" (UID: "b04e3abe-c8da-4123-ae78-3d978812545d") : secret "dns-default-metrics-tls" not found
Jul 07 17:08:37.305578 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:37.305494    2579 generic.go:356] "Generic (PLEG): container finished" podID="1c55e58a-1b4a-4643-923c-c235327231ed" containerID="2ff1963e18258b18a637ec127484bfa4f4aa898f7a50c1bcb9bfe9b2545ab9f8" exitCode=0
Jul 07 17:08:37.306143 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:37.305581    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4h5b6" event={"ID":"1c55e58a-1b4a-4643-923c-c235327231ed","Type":"ContainerDied","Data":"2ff1963e18258b18a637ec127484bfa4f4aa898f7a50c1bcb9bfe9b2545ab9f8"}
Jul 07 17:08:37.307272 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:37.307244    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-h7sqt" event={"ID":"72d06848-e504-4952-953f-02b4800ddf45","Type":"ContainerStarted","Data":"cdf7ca5549acaa3c291fc73e3fd3cfaa3570c42e706ccbcf3499de65e921dc74"}
Jul 07 17:08:37.354019 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:37.353958    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-h7sqt" podStartSLOduration=1.19896102 podStartE2EDuration="5.353940992s" totalImagesPullingTime="4.154979972s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:08:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:08:37.353418747 +0000 UTC m=+40.849497256" watchObservedRunningTime="2026-07-07 17:08:37.353940992 +0000 UTC m=+40.850019500"
Jul 07 17:08:38.312204 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:38.312172    2579 generic.go:356] "Generic (PLEG): container finished" podID="1c55e58a-1b4a-4643-923c-c235327231ed" containerID="91bc315bee945955f19f014fcf89609bc3cbcc769c7abe5455e5b59bf1376877" exitCode=0
Jul 07 17:08:38.312639 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:38.312261    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4h5b6" event={"ID":"1c55e58a-1b4a-4643-923c-c235327231ed","Type":"ContainerDied","Data":"91bc315bee945955f19f014fcf89609bc3cbcc769c7abe5455e5b59bf1376877"}
Jul 07 17:08:39.317457 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:39.317424    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4h5b6" event={"ID":"1c55e58a-1b4a-4643-923c-c235327231ed","Type":"ContainerStarted","Data":"03bd3ac9cb65494155cf32892ff18276d458df30d8d79a5b0a58fa10a63b7833"}
Jul 07 17:08:39.355460 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:39.355408    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-4h5b6" podStartSLOduration=7.811454288 podStartE2EDuration="41.355393096s" totalImagesPullingTime="33.543938808s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:07:58 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:08:39.353091843 +0000 UTC m=+42.849170345" watchObservedRunningTime="2026-07-07 17:08:39.355393096 +0000 UTC m=+42.851471603"
Jul 07 17:08:40.098541 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:40.098505    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b04e3abe-c8da-4123-ae78-3d978812545d-metrics-tls\") pod \"dns-default-nbkp6\" (UID: \"b04e3abe-c8da-4123-ae78-3d978812545d\") " pod="openshift-dns/dns-default-nbkp6"
Jul 07 17:08:40.098672 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:40.098645    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 17:08:40.098719 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:40.098705    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b04e3abe-c8da-4123-ae78-3d978812545d-metrics-tls podName:b04e3abe-c8da-4123-ae78-3d978812545d nodeName:}" failed. No retries permitted until 2026-07-07 17:08:48.098688799 +0000 UTC m=+51.594767292 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b04e3abe-c8da-4123-ae78-3d978812545d-metrics-tls") pod "dns-default-nbkp6" (UID: "b04e3abe-c8da-4123-ae78-3d978812545d") : secret "dns-default-metrics-tls" not found
Jul 07 17:08:47.342199 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.342172    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-thcnc"]
Jul 07 17:08:47.358098 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.358073    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-thcnc"
Jul 07 17:08:47.361500 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.361471    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-d95rr\""
Jul 07 17:08:47.361500 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.361494    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 07 17:08:47.362223 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.362210    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 07 17:08:47.367124 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.367103    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-thcnc"]
Jul 07 17:08:47.502004 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.501978    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-b4c4c8fd5-x4qmh"]
Jul 07 17:08:47.523339 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.523312    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-v4z9l"]
Jul 07 17:08:47.523451 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.523435    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b4c4c8fd5-x4qmh"
Jul 07 17:08:47.532595 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.532576    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-n59wn\""
Jul 07 17:08:47.532680 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.532601    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 07 17:08:47.532680 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.532612    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 07 17:08:47.532680 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.532601    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 07 17:08:47.541639 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.541622    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-v4z9l"]
Jul 07 17:08:47.541713 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.541695    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-v4z9l"
Jul 07 17:08:47.545151 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.545130    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 07 17:08:47.545380 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.545364    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 07 17:08:47.545637 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.545497    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 07 17:08:47.545637 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.545374    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 07 17:08:47.545637 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.545577    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-kh2cc\""
Jul 07 17:08:47.546515 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.546498    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-b4c4c8fd5-x4qmh"]
Jul 07 17:08:47.551509 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.551490    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k66f6\" (UniqueName: \"kubernetes.io/projected/15180ffd-916f-4516-8bb4-967a3701c482-kube-api-access-k66f6\") pod \"volume-data-source-validator-5d456fdbcd-thcnc\" (UID: \"15180ffd-916f-4516-8bb4-967a3701c482\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-thcnc"
Jul 07 17:08:47.563061 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.563045    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 07 17:08:47.652055 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.652033    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-ca-trust-extracted\") pod \"image-registry-b4c4c8fd5-x4qmh\" (UID: \"9535133b-ad4d-42dd-a756-c4d4c1f1fdc5\") " pod="openshift-image-registry/image-registry-b4c4c8fd5-x4qmh"
Jul 07 17:08:47.652136 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.652079    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k66f6\" (UniqueName: \"kubernetes.io/projected/15180ffd-916f-4516-8bb4-967a3701c482-kube-api-access-k66f6\") pod \"volume-data-source-validator-5d456fdbcd-thcnc\" (UID: \"15180ffd-916f-4516-8bb4-967a3701c482\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-thcnc"
Jul 07 17:08:47.652136 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.652098    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-image-registry-private-configuration\") pod \"image-registry-b4c4c8fd5-x4qmh\" (UID: \"9535133b-ad4d-42dd-a756-c4d4c1f1fdc5\") " pod="openshift-image-registry/image-registry-b4c4c8fd5-x4qmh"
Jul 07 17:08:47.652136 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.652122    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-registry-tls\") pod \"image-registry-b4c4c8fd5-x4qmh\" (UID: \"9535133b-ad4d-42dd-a756-c4d4c1f1fdc5\") " pod="openshift-image-registry/image-registry-b4c4c8fd5-x4qmh"
Jul 07 17:08:47.652256 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.652139    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-trusted-ca\") pod \"image-registry-b4c4c8fd5-x4qmh\" (UID: \"9535133b-ad4d-42dd-a756-c4d4c1f1fdc5\") " pod="openshift-image-registry/image-registry-b4c4c8fd5-x4qmh"
Jul 07 17:08:47.652256 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.652153    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-installation-pull-secrets\") pod \"image-registry-b4c4c8fd5-x4qmh\" (UID: \"9535133b-ad4d-42dd-a756-c4d4c1f1fdc5\") " pod="openshift-image-registry/image-registry-b4c4c8fd5-x4qmh"
Jul 07 17:08:47.652256 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.652186    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-registry-certificates\") pod \"image-registry-b4c4c8fd5-x4qmh\" (UID: \"9535133b-ad4d-42dd-a756-c4d4c1f1fdc5\") " pod="openshift-image-registry/image-registry-b4c4c8fd5-x4qmh"
Jul 07 17:08:47.652256 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.652229    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bb66b2cc-3724-4d3b-a6ff-adc97538bf77-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-v4z9l\" (UID: \"bb66b2cc-3724-4d3b-a6ff-adc97538bf77\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-v4z9l"
Jul 07 17:08:47.652385 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.652270    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bb66b2cc-3724-4d3b-a6ff-adc97538bf77-config\") pod \"service-ca-operator-5d46d9ffd4-v4z9l\" (UID: \"bb66b2cc-3724-4d3b-a6ff-adc97538bf77\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-v4z9l"
Jul 07 17:08:47.652385 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.652293    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfvp5\" (UniqueName: \"kubernetes.io/projected/bb66b2cc-3724-4d3b-a6ff-adc97538bf77-kube-api-access-cfvp5\") pod \"service-ca-operator-5d46d9ffd4-v4z9l\" (UID: \"bb66b2cc-3724-4d3b-a6ff-adc97538bf77\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-v4z9l"
Jul 07 17:08:47.652385 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.652311    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-bound-sa-token\") pod \"image-registry-b4c4c8fd5-x4qmh\" (UID: \"9535133b-ad4d-42dd-a756-c4d4c1f1fdc5\") " pod="openshift-image-registry/image-registry-b4c4c8fd5-x4qmh"
Jul 07 17:08:47.652385 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.652326    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jbvf\" (UniqueName: \"kubernetes.io/projected/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-kube-api-access-9jbvf\") pod \"image-registry-b4c4c8fd5-x4qmh\" (UID: \"9535133b-ad4d-42dd-a756-c4d4c1f1fdc5\") " pod="openshift-image-registry/image-registry-b4c4c8fd5-x4qmh"
Jul 07 17:08:47.663787 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.663762    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k66f6\" (UniqueName: \"kubernetes.io/projected/15180ffd-916f-4516-8bb4-967a3701c482-kube-api-access-k66f6\") pod \"volume-data-source-validator-5d456fdbcd-thcnc\" (UID: \"15180ffd-916f-4516-8bb4-967a3701c482\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-thcnc"
Jul 07 17:08:47.666499 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.666478    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-thcnc"
Jul 07 17:08:47.753404 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.753379    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bb66b2cc-3724-4d3b-a6ff-adc97538bf77-config\") pod \"service-ca-operator-5d46d9ffd4-v4z9l\" (UID: \"bb66b2cc-3724-4d3b-a6ff-adc97538bf77\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-v4z9l"
Jul 07 17:08:47.753477 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.753423    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfvp5\" (UniqueName: \"kubernetes.io/projected/bb66b2cc-3724-4d3b-a6ff-adc97538bf77-kube-api-access-cfvp5\") pod \"service-ca-operator-5d46d9ffd4-v4z9l\" (UID: \"bb66b2cc-3724-4d3b-a6ff-adc97538bf77\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-v4z9l"
Jul 07 17:08:47.753477 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.753459    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-bound-sa-token\") pod \"image-registry-b4c4c8fd5-x4qmh\" (UID: \"9535133b-ad4d-42dd-a756-c4d4c1f1fdc5\") " pod="openshift-image-registry/image-registry-b4c4c8fd5-x4qmh"
Jul 07 17:08:47.753657 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.753488    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jbvf\" (UniqueName: \"kubernetes.io/projected/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-kube-api-access-9jbvf\") pod \"image-registry-b4c4c8fd5-x4qmh\" (UID: \"9535133b-ad4d-42dd-a756-c4d4c1f1fdc5\") " pod="openshift-image-registry/image-registry-b4c4c8fd5-x4qmh"
Jul 07 17:08:47.753657 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.753530    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-ca-trust-extracted\") pod \"image-registry-b4c4c8fd5-x4qmh\" (UID: \"9535133b-ad4d-42dd-a756-c4d4c1f1fdc5\") " pod="openshift-image-registry/image-registry-b4c4c8fd5-x4qmh"
Jul 07 17:08:47.753657 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.753604    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-image-registry-private-configuration\") pod \"image-registry-b4c4c8fd5-x4qmh\" (UID: \"9535133b-ad4d-42dd-a756-c4d4c1f1fdc5\") " pod="openshift-image-registry/image-registry-b4c4c8fd5-x4qmh"
Jul 07 17:08:47.753657 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.753629    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-registry-tls\") pod \"image-registry-b4c4c8fd5-x4qmh\" (UID: \"9535133b-ad4d-42dd-a756-c4d4c1f1fdc5\") " pod="openshift-image-registry/image-registry-b4c4c8fd5-x4qmh"
Jul 07 17:08:47.753657 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.753652    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-trusted-ca\") pod \"image-registry-b4c4c8fd5-x4qmh\" (UID: \"9535133b-ad4d-42dd-a756-c4d4c1f1fdc5\") " pod="openshift-image-registry/image-registry-b4c4c8fd5-x4qmh"
Jul 07 17:08:47.753868 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.753679    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-installation-pull-secrets\") pod \"image-registry-b4c4c8fd5-x4qmh\" (UID: \"9535133b-ad4d-42dd-a756-c4d4c1f1fdc5\") " pod="openshift-image-registry/image-registry-b4c4c8fd5-x4qmh"
Jul 07 17:08:47.753868 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.753717    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-registry-certificates\") pod \"image-registry-b4c4c8fd5-x4qmh\" (UID: \"9535133b-ad4d-42dd-a756-c4d4c1f1fdc5\") " pod="openshift-image-registry/image-registry-b4c4c8fd5-x4qmh"
Jul 07 17:08:47.753868 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.753748    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bb66b2cc-3724-4d3b-a6ff-adc97538bf77-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-v4z9l\" (UID: \"bb66b2cc-3724-4d3b-a6ff-adc97538bf77\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-v4z9l"
Jul 07 17:08:47.754187 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.754152    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bb66b2cc-3724-4d3b-a6ff-adc97538bf77-config\") pod \"service-ca-operator-5d46d9ffd4-v4z9l\" (UID: \"bb66b2cc-3724-4d3b-a6ff-adc97538bf77\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-v4z9l"
Jul 07 17:08:47.754270 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:47.754248    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 17:08:47.754270 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:47.754267    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b4c4c8fd5-x4qmh: secret "image-registry-tls" not found
Jul 07 17:08:47.754371 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:47.754331    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-registry-tls podName:9535133b-ad4d-42dd-a756-c4d4c1f1fdc5 nodeName:}" failed. No retries permitted until 2026-07-07 17:08:48.254312058 +0000 UTC m=+51.750390559 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-registry-tls") pod "image-registry-b4c4c8fd5-x4qmh" (UID: "9535133b-ad4d-42dd-a756-c4d4c1f1fdc5") : secret "image-registry-tls" not found
Jul 07 17:08:47.756333 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.754495    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-ca-trust-extracted\") pod \"image-registry-b4c4c8fd5-x4qmh\" (UID: \"9535133b-ad4d-42dd-a756-c4d4c1f1fdc5\") " pod="openshift-image-registry/image-registry-b4c4c8fd5-x4qmh"
Jul 07 17:08:47.756333 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.755196    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-registry-certificates\") pod \"image-registry-b4c4c8fd5-x4qmh\" (UID: \"9535133b-ad4d-42dd-a756-c4d4c1f1fdc5\") " pod="openshift-image-registry/image-registry-b4c4c8fd5-x4qmh"
Jul 07 17:08:47.756735 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.756712    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-trusted-ca\") pod \"image-registry-b4c4c8fd5-x4qmh\" (UID: \"9535133b-ad4d-42dd-a756-c4d4c1f1fdc5\") " pod="openshift-image-registry/image-registry-b4c4c8fd5-x4qmh"
Jul 07 17:08:47.757278 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.757253    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-image-registry-private-configuration\") pod \"image-registry-b4c4c8fd5-x4qmh\" (UID: \"9535133b-ad4d-42dd-a756-c4d4c1f1fdc5\") " pod="openshift-image-registry/image-registry-b4c4c8fd5-x4qmh"
Jul 07 17:08:47.757483 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.757463    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bb66b2cc-3724-4d3b-a6ff-adc97538bf77-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-v4z9l\" (UID: \"bb66b2cc-3724-4d3b-a6ff-adc97538bf77\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-v4z9l"
Jul 07 17:08:47.757869 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.757850    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-installation-pull-secrets\") pod \"image-registry-b4c4c8fd5-x4qmh\" (UID: \"9535133b-ad4d-42dd-a756-c4d4c1f1fdc5\") " pod="openshift-image-registry/image-registry-b4c4c8fd5-x4qmh"
Jul 07 17:08:47.764534 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.764514    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfvp5\" (UniqueName: \"kubernetes.io/projected/bb66b2cc-3724-4d3b-a6ff-adc97538bf77-kube-api-access-cfvp5\") pod \"service-ca-operator-5d46d9ffd4-v4z9l\" (UID: \"bb66b2cc-3724-4d3b-a6ff-adc97538bf77\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-v4z9l"
Jul 07 17:08:47.768649 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.768622    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jbvf\" (UniqueName: \"kubernetes.io/projected/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-kube-api-access-9jbvf\") pod \"image-registry-b4c4c8fd5-x4qmh\" (UID: \"9535133b-ad4d-42dd-a756-c4d4c1f1fdc5\") " pod="openshift-image-registry/image-registry-b4c4c8fd5-x4qmh"
Jul 07 17:08:47.768724 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.768666    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-bound-sa-token\") pod \"image-registry-b4c4c8fd5-x4qmh\" (UID: \"9535133b-ad4d-42dd-a756-c4d4c1f1fdc5\") " pod="openshift-image-registry/image-registry-b4c4c8fd5-x4qmh"
Jul 07 17:08:47.800837 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.800816    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-thcnc"]
Jul 07 17:08:47.804204 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:08:47.804179    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15180ffd_916f_4516_8bb4_967a3701c482.slice/crio-e480483c880266870190b8e32650df99d9dbf23f1f1dcfd0e5274bee9e351369 WatchSource:0}: Error finding container e480483c880266870190b8e32650df99d9dbf23f1f1dcfd0e5274bee9e351369: Status 404 returned error can't find the container with id e480483c880266870190b8e32650df99d9dbf23f1f1dcfd0e5274bee9e351369
Jul 07 17:08:47.851257 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:47.851237    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-v4z9l"
Jul 07 17:08:48.006953 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:48.006928    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-v4z9l"]
Jul 07 17:08:48.009966 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:08:48.009939    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb66b2cc_3724_4d3b_a6ff_adc97538bf77.slice/crio-cc095ae8307d28aca0c0e82365395fd9dfe389d418271887ff78308983dd7751 WatchSource:0}: Error finding container cc095ae8307d28aca0c0e82365395fd9dfe389d418271887ff78308983dd7751: Status 404 returned error can't find the container with id cc095ae8307d28aca0c0e82365395fd9dfe389d418271887ff78308983dd7751
Jul 07 17:08:48.156815 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:48.156788    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b04e3abe-c8da-4123-ae78-3d978812545d-metrics-tls\") pod \"dns-default-nbkp6\" (UID: \"b04e3abe-c8da-4123-ae78-3d978812545d\") " pod="openshift-dns/dns-default-nbkp6"
Jul 07 17:08:48.156958 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:48.156944    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 17:08:48.157013 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:48.156993    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b04e3abe-c8da-4123-ae78-3d978812545d-metrics-tls podName:b04e3abe-c8da-4123-ae78-3d978812545d nodeName:}" failed. No retries permitted until 2026-07-07 17:09:04.156980524 +0000 UTC m=+67.653059014 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b04e3abe-c8da-4123-ae78-3d978812545d-metrics-tls") pod "dns-default-nbkp6" (UID: "b04e3abe-c8da-4123-ae78-3d978812545d") : secret "dns-default-metrics-tls" not found
Jul 07 17:08:48.258015 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:48.257969    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-registry-tls\") pod \"image-registry-b4c4c8fd5-x4qmh\" (UID: \"9535133b-ad4d-42dd-a756-c4d4c1f1fdc5\") " pod="openshift-image-registry/image-registry-b4c4c8fd5-x4qmh"
Jul 07 17:08:48.258091 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:48.258063    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 17:08:48.258091 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:48.258074    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b4c4c8fd5-x4qmh: secret "image-registry-tls" not found
Jul 07 17:08:48.258180 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:48.258113    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-registry-tls podName:9535133b-ad4d-42dd-a756-c4d4c1f1fdc5 nodeName:}" failed. No retries permitted until 2026-07-07 17:08:49.258102326 +0000 UTC m=+52.754180815 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-registry-tls") pod "image-registry-b4c4c8fd5-x4qmh" (UID: "9535133b-ad4d-42dd-a756-c4d4c1f1fdc5") : secret "image-registry-tls" not found
Jul 07 17:08:48.336636 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:48.336608    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-thcnc" event={"ID":"15180ffd-916f-4516-8bb4-967a3701c482","Type":"ContainerStarted","Data":"e480483c880266870190b8e32650df99d9dbf23f1f1dcfd0e5274bee9e351369"}
Jul 07 17:08:48.337648 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:48.337629    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-v4z9l" event={"ID":"bb66b2cc-3724-4d3b-a6ff-adc97538bf77","Type":"ContainerStarted","Data":"cc095ae8307d28aca0c0e82365395fd9dfe389d418271887ff78308983dd7751"}
Jul 07 17:08:49.265416 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:49.265385    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-registry-tls\") pod \"image-registry-b4c4c8fd5-x4qmh\" (UID: \"9535133b-ad4d-42dd-a756-c4d4c1f1fdc5\") " pod="openshift-image-registry/image-registry-b4c4c8fd5-x4qmh"
Jul 07 17:08:49.265882 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:49.265551    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 17:08:49.265882 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:49.265572    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b4c4c8fd5-x4qmh: secret "image-registry-tls" not found
Jul 07 17:08:49.265882 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:49.265636    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-registry-tls podName:9535133b-ad4d-42dd-a756-c4d4c1f1fdc5 nodeName:}" failed. No retries permitted until 2026-07-07 17:08:51.265619513 +0000 UTC m=+54.761698020 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-registry-tls") pod "image-registry-b4c4c8fd5-x4qmh" (UID: "9535133b-ad4d-42dd-a756-c4d4c1f1fdc5") : secret "image-registry-tls" not found
Jul 07 17:08:50.344472 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:50.344396    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-thcnc" event={"ID":"15180ffd-916f-4516-8bb4-967a3701c482","Type":"ContainerStarted","Data":"5b4bf1d51fa758b8ce2865b59ee48a5e384388f036cb289b9d34224ec4b62e78"}
Jul 07 17:08:50.364782 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:50.364734    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-thcnc" podStartSLOduration=1.105963605 podStartE2EDuration="3.364719483s" totalImagesPullingTime="2.258755878s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:08:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:08:50.363819192 +0000 UTC m=+53.859897700" watchObservedRunningTime="2026-07-07 17:08:50.364719483 +0000 UTC m=+53.860798045"
Jul 07 17:08:51.279506 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:51.279465    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-registry-tls\") pod \"image-registry-b4c4c8fd5-x4qmh\" (UID: \"9535133b-ad4d-42dd-a756-c4d4c1f1fdc5\") " pod="openshift-image-registry/image-registry-b4c4c8fd5-x4qmh"
Jul 07 17:08:51.279670 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:51.279639    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 17:08:51.279670 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:51.279659    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b4c4c8fd5-x4qmh: secret "image-registry-tls" not found
Jul 07 17:08:51.279786 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:51.279731    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-registry-tls podName:9535133b-ad4d-42dd-a756-c4d4c1f1fdc5 nodeName:}" failed. No retries permitted until 2026-07-07 17:08:55.279709542 +0000 UTC m=+58.775788041 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-registry-tls") pod "image-registry-b4c4c8fd5-x4qmh" (UID: "9535133b-ad4d-42dd-a756-c4d4c1f1fdc5") : secret "image-registry-tls" not found
Jul 07 17:08:51.599242 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:51.598453    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-ph5mc"]
Jul 07 17:08:51.617976 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:51.617951    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ph5mc"]
Jul 07 17:08:51.618093 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:51.618067    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ph5mc"
Jul 07 17:08:51.621903 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:51.621865    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 07 17:08:51.622026 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:51.622002    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 07 17:08:51.622081 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:51.622031    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 07 17:08:51.622792 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:51.622777    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-kb2xx\""
Jul 07 17:08:51.623104 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:51.623089    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 07 17:08:51.701973 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:51.701952    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/299d1c5f-8255-4e02-9cef-7a80ce5f88d7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ph5mc\" (UID: \"299d1c5f-8255-4e02-9cef-7a80ce5f88d7\") " pod="openshift-insights/insights-runtime-extractor-ph5mc"
Jul 07 17:08:51.702049 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:51.702025    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/299d1c5f-8255-4e02-9cef-7a80ce5f88d7-data-volume\") pod \"insights-runtime-extractor-ph5mc\" (UID: \"299d1c5f-8255-4e02-9cef-7a80ce5f88d7\") " pod="openshift-insights/insights-runtime-extractor-ph5mc"
Jul 07 17:08:51.702049 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:51.702044    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/299d1c5f-8255-4e02-9cef-7a80ce5f88d7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ph5mc\" (UID: \"299d1c5f-8255-4e02-9cef-7a80ce5f88d7\") " pod="openshift-insights/insights-runtime-extractor-ph5mc"
Jul 07 17:08:51.702117 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:51.702067    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sm8dk\" (UniqueName: \"kubernetes.io/projected/299d1c5f-8255-4e02-9cef-7a80ce5f88d7-kube-api-access-sm8dk\") pod \"insights-runtime-extractor-ph5mc\" (UID: \"299d1c5f-8255-4e02-9cef-7a80ce5f88d7\") " pod="openshift-insights/insights-runtime-extractor-ph5mc"
Jul 07 17:08:51.702117 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:51.702102    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/299d1c5f-8255-4e02-9cef-7a80ce5f88d7-crio-socket\") pod \"insights-runtime-extractor-ph5mc\" (UID: \"299d1c5f-8255-4e02-9cef-7a80ce5f88d7\") " pod="openshift-insights/insights-runtime-extractor-ph5mc"
Jul 07 17:08:51.802520 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:51.802498    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/299d1c5f-8255-4e02-9cef-7a80ce5f88d7-crio-socket\") pod \"insights-runtime-extractor-ph5mc\" (UID: \"299d1c5f-8255-4e02-9cef-7a80ce5f88d7\") " pod="openshift-insights/insights-runtime-extractor-ph5mc"
Jul 07 17:08:51.802633 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:51.802549    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/299d1c5f-8255-4e02-9cef-7a80ce5f88d7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ph5mc\" (UID: \"299d1c5f-8255-4e02-9cef-7a80ce5f88d7\") " pod="openshift-insights/insights-runtime-extractor-ph5mc"
Jul 07 17:08:51.802633 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:51.802613    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/299d1c5f-8255-4e02-9cef-7a80ce5f88d7-crio-socket\") pod \"insights-runtime-extractor-ph5mc\" (UID: \"299d1c5f-8255-4e02-9cef-7a80ce5f88d7\") " pod="openshift-insights/insights-runtime-extractor-ph5mc"
Jul 07 17:08:51.802743 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:51.802620    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 17:08:51.802743 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:51.802652    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/299d1c5f-8255-4e02-9cef-7a80ce5f88d7-data-volume\") pod \"insights-runtime-extractor-ph5mc\" (UID: \"299d1c5f-8255-4e02-9cef-7a80ce5f88d7\") " pod="openshift-insights/insights-runtime-extractor-ph5mc"
Jul 07 17:08:51.802743 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:51.802682    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/299d1c5f-8255-4e02-9cef-7a80ce5f88d7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ph5mc\" (UID: \"299d1c5f-8255-4e02-9cef-7a80ce5f88d7\") " pod="openshift-insights/insights-runtime-extractor-ph5mc"
Jul 07 17:08:51.802743 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:51.802696    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/299d1c5f-8255-4e02-9cef-7a80ce5f88d7-kube-rbac-proxy-cm podName:299d1c5f-8255-4e02-9cef-7a80ce5f88d7 nodeName:}" failed. No retries permitted until 2026-07-07 17:08:52.302676974 +0000 UTC m=+55.798755463 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/299d1c5f-8255-4e02-9cef-7a80ce5f88d7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ph5mc" (UID: "299d1c5f-8255-4e02-9cef-7a80ce5f88d7") : configmap "kube-rbac-proxy" not found
Jul 07 17:08:51.802743 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:51.802725    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sm8dk\" (UniqueName: \"kubernetes.io/projected/299d1c5f-8255-4e02-9cef-7a80ce5f88d7-kube-api-access-sm8dk\") pod \"insights-runtime-extractor-ph5mc\" (UID: \"299d1c5f-8255-4e02-9cef-7a80ce5f88d7\") " pod="openshift-insights/insights-runtime-extractor-ph5mc"
Jul 07 17:08:51.802978 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:51.802777    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 17:08:51.802978 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:51.802836    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/299d1c5f-8255-4e02-9cef-7a80ce5f88d7-insights-runtime-extractor-tls podName:299d1c5f-8255-4e02-9cef-7a80ce5f88d7 nodeName:}" failed. No retries permitted until 2026-07-07 17:08:52.302820206 +0000 UTC m=+55.798898699 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/299d1c5f-8255-4e02-9cef-7a80ce5f88d7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ph5mc" (UID: "299d1c5f-8255-4e02-9cef-7a80ce5f88d7") : secret "insights-runtime-extractor-tls" not found
Jul 07 17:08:51.803061 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:51.803042    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/299d1c5f-8255-4e02-9cef-7a80ce5f88d7-data-volume\") pod \"insights-runtime-extractor-ph5mc\" (UID: \"299d1c5f-8255-4e02-9cef-7a80ce5f88d7\") " pod="openshift-insights/insights-runtime-extractor-ph5mc"
Jul 07 17:08:51.811943 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:51.811921    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sm8dk\" (UniqueName: \"kubernetes.io/projected/299d1c5f-8255-4e02-9cef-7a80ce5f88d7-kube-api-access-sm8dk\") pod \"insights-runtime-extractor-ph5mc\" (UID: \"299d1c5f-8255-4e02-9cef-7a80ce5f88d7\") " pod="openshift-insights/insights-runtime-extractor-ph5mc"
Jul 07 17:08:52.304813 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:52.304782    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/299d1c5f-8255-4e02-9cef-7a80ce5f88d7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ph5mc\" (UID: \"299d1c5f-8255-4e02-9cef-7a80ce5f88d7\") " pod="openshift-insights/insights-runtime-extractor-ph5mc"
Jul 07 17:08:52.305036 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:52.304873    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/299d1c5f-8255-4e02-9cef-7a80ce5f88d7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ph5mc\" (UID: \"299d1c5f-8255-4e02-9cef-7a80ce5f88d7\") " pod="openshift-insights/insights-runtime-extractor-ph5mc"
Jul 07 17:08:52.305036 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:52.304910    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 17:08:52.305036 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:52.304964    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/299d1c5f-8255-4e02-9cef-7a80ce5f88d7-kube-rbac-proxy-cm podName:299d1c5f-8255-4e02-9cef-7a80ce5f88d7 nodeName:}" failed. No retries permitted until 2026-07-07 17:08:53.304949737 +0000 UTC m=+56.801028223 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/299d1c5f-8255-4e02-9cef-7a80ce5f88d7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ph5mc" (UID: "299d1c5f-8255-4e02-9cef-7a80ce5f88d7") : configmap "kube-rbac-proxy" not found
Jul 07 17:08:52.305154 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:52.305042    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 17:08:52.305154 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:52.305120    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/299d1c5f-8255-4e02-9cef-7a80ce5f88d7-insights-runtime-extractor-tls podName:299d1c5f-8255-4e02-9cef-7a80ce5f88d7 nodeName:}" failed. No retries permitted until 2026-07-07 17:08:53.305101943 +0000 UTC m=+56.801180432 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/299d1c5f-8255-4e02-9cef-7a80ce5f88d7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ph5mc" (UID: "299d1c5f-8255-4e02-9cef-7a80ce5f88d7") : secret "insights-runtime-extractor-tls" not found
Jul 07 17:08:52.350514 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:52.350490    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-v4z9l" event={"ID":"bb66b2cc-3724-4d3b-a6ff-adc97538bf77","Type":"ContainerStarted","Data":"e0109bd971ff8c0dd5042849e1efd7165af040e78e9f6f5bc07efc8575d92de8"}
Jul 07 17:08:52.378291 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:52.378253    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-v4z9l" podStartSLOduration=1.710026675 podStartE2EDuration="5.37823969s" totalImagesPullingTime="3.668213015s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:08:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:08:52.377636053 +0000 UTC m=+55.873714564" watchObservedRunningTime="2026-07-07 17:08:52.37823969 +0000 UTC m=+55.874318198"
Jul 07 17:08:53.313216 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:53.313188    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/299d1c5f-8255-4e02-9cef-7a80ce5f88d7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ph5mc\" (UID: \"299d1c5f-8255-4e02-9cef-7a80ce5f88d7\") " pod="openshift-insights/insights-runtime-extractor-ph5mc"
Jul 07 17:08:53.313657 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:53.313263    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/299d1c5f-8255-4e02-9cef-7a80ce5f88d7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ph5mc\" (UID: \"299d1c5f-8255-4e02-9cef-7a80ce5f88d7\") " pod="openshift-insights/insights-runtime-extractor-ph5mc"
Jul 07 17:08:53.313657 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:53.313315    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 17:08:53.313657 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:53.313358    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 17:08:53.313657 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:53.313373    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/299d1c5f-8255-4e02-9cef-7a80ce5f88d7-kube-rbac-proxy-cm podName:299d1c5f-8255-4e02-9cef-7a80ce5f88d7 nodeName:}" failed. No retries permitted until 2026-07-07 17:08:55.3133566 +0000 UTC m=+58.809435103 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/299d1c5f-8255-4e02-9cef-7a80ce5f88d7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ph5mc" (UID: "299d1c5f-8255-4e02-9cef-7a80ce5f88d7") : configmap "kube-rbac-proxy" not found
Jul 07 17:08:53.313657 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:53.313407    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/299d1c5f-8255-4e02-9cef-7a80ce5f88d7-insights-runtime-extractor-tls podName:299d1c5f-8255-4e02-9cef-7a80ce5f88d7 nodeName:}" failed. No retries permitted until 2026-07-07 17:08:55.313395293 +0000 UTC m=+58.809473794 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/299d1c5f-8255-4e02-9cef-7a80ce5f88d7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ph5mc" (UID: "299d1c5f-8255-4e02-9cef-7a80ce5f88d7") : secret "insights-runtime-extractor-tls" not found
Jul 07 17:08:53.372702 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:53.372683    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-66zv2_ecf1ac63-1ea5-4590-8d60-b199e5eff959/dns-node-resolver/0.log"
Jul 07 17:08:54.371183 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:54.371154    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-87n7p_187a8f3e-75c8-461a-a0f0-f3e3723d3a34/node-ca/0.log"
Jul 07 17:08:55.329064 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:55.329030    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-registry-tls\") pod \"image-registry-b4c4c8fd5-x4qmh\" (UID: \"9535133b-ad4d-42dd-a756-c4d4c1f1fdc5\") " pod="openshift-image-registry/image-registry-b4c4c8fd5-x4qmh"
Jul 07 17:08:55.329208 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:55.329076    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/299d1c5f-8255-4e02-9cef-7a80ce5f88d7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ph5mc\" (UID: \"299d1c5f-8255-4e02-9cef-7a80ce5f88d7\") " pod="openshift-insights/insights-runtime-extractor-ph5mc"
Jul 07 17:08:55.329208 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:55.329155    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/299d1c5f-8255-4e02-9cef-7a80ce5f88d7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ph5mc\" (UID: \"299d1c5f-8255-4e02-9cef-7a80ce5f88d7\") " pod="openshift-insights/insights-runtime-extractor-ph5mc"
Jul 07 17:08:55.329208 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:55.329183    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 17:08:55.329305 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:55.329220    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 17:08:55.329305 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:55.329242    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/299d1c5f-8255-4e02-9cef-7a80ce5f88d7-insights-runtime-extractor-tls podName:299d1c5f-8255-4e02-9cef-7a80ce5f88d7 nodeName:}" failed. No retries permitted until 2026-07-07 17:08:59.329225127 +0000 UTC m=+62.825303632 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/299d1c5f-8255-4e02-9cef-7a80ce5f88d7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ph5mc" (UID: "299d1c5f-8255-4e02-9cef-7a80ce5f88d7") : secret "insights-runtime-extractor-tls" not found
Jul 07 17:08:55.329305 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:55.329259    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/299d1c5f-8255-4e02-9cef-7a80ce5f88d7-kube-rbac-proxy-cm podName:299d1c5f-8255-4e02-9cef-7a80ce5f88d7 nodeName:}" failed. No retries permitted until 2026-07-07 17:08:59.32924853 +0000 UTC m=+62.825327016 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/299d1c5f-8255-4e02-9cef-7a80ce5f88d7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ph5mc" (UID: "299d1c5f-8255-4e02-9cef-7a80ce5f88d7") : configmap "kube-rbac-proxy" not found
Jul 07 17:08:55.329406 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:55.329315    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 17:08:55.329406 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:55.329333    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b4c4c8fd5-x4qmh: secret "image-registry-tls" not found
Jul 07 17:08:55.329406 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:55.329377    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-registry-tls podName:9535133b-ad4d-42dd-a756-c4d4c1f1fdc5 nodeName:}" failed. No retries permitted until 2026-07-07 17:09:03.329364701 +0000 UTC m=+66.825443186 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-registry-tls") pod "image-registry-b4c4c8fd5-x4qmh" (UID: "9535133b-ad4d-42dd-a756-c4d4c1f1fdc5") : secret "image-registry-tls" not found
Jul 07 17:08:55.372135 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:55.372114    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-h7sqt_72d06848-e504-4952-953f-02b4800ddf45/serve-healthcheck-canary/0.log"
Jul 07 17:08:55.514993 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:55.514972    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-vqfmq"]
Jul 07 17:08:55.517251 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:55.517237    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-vqfmq"
Jul 07 17:08:55.519837 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:55.519817    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-9g5jn\""
Jul 07 17:08:55.520545 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:55.520530    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 07 17:08:55.520602 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:55.520544    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 07 17:08:55.520647 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:55.520630    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 07 17:08:55.520853 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:55.520838    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 07 17:08:55.531523 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:55.531498    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/9503836b-029a-462a-bea1-ddc8684b2b65-signing-key\") pod \"service-ca-758f599c5-vqfmq\" (UID: \"9503836b-029a-462a-bea1-ddc8684b2b65\") " pod="openshift-service-ca/service-ca-758f599c5-vqfmq"
Jul 07 17:08:55.531623 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:55.531598    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmp2q\" (UniqueName: \"kubernetes.io/projected/9503836b-029a-462a-bea1-ddc8684b2b65-kube-api-access-hmp2q\") pod \"service-ca-758f599c5-vqfmq\" (UID: \"9503836b-029a-462a-bea1-ddc8684b2b65\") " pod="openshift-service-ca/service-ca-758f599c5-vqfmq"
Jul 07 17:08:55.531722 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:55.531701    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/9503836b-029a-462a-bea1-ddc8684b2b65-signing-cabundle\") pod \"service-ca-758f599c5-vqfmq\" (UID: \"9503836b-029a-462a-bea1-ddc8684b2b65\") " pod="openshift-service-ca/service-ca-758f599c5-vqfmq"
Jul 07 17:08:55.532794 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:55.532775    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-vqfmq"]
Jul 07 17:08:55.631959 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:55.631910    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/9503836b-029a-462a-bea1-ddc8684b2b65-signing-key\") pod \"service-ca-758f599c5-vqfmq\" (UID: \"9503836b-029a-462a-bea1-ddc8684b2b65\") " pod="openshift-service-ca/service-ca-758f599c5-vqfmq"
Jul 07 17:08:55.631959 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:55.631953    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmp2q\" (UniqueName: \"kubernetes.io/projected/9503836b-029a-462a-bea1-ddc8684b2b65-kube-api-access-hmp2q\") pod \"service-ca-758f599c5-vqfmq\" (UID: \"9503836b-029a-462a-bea1-ddc8684b2b65\") " pod="openshift-service-ca/service-ca-758f599c5-vqfmq"
Jul 07 17:08:55.632100 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:55.632025    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/9503836b-029a-462a-bea1-ddc8684b2b65-signing-cabundle\") pod \"service-ca-758f599c5-vqfmq\" (UID: \"9503836b-029a-462a-bea1-ddc8684b2b65\") " pod="openshift-service-ca/service-ca-758f599c5-vqfmq"
Jul 07 17:08:55.632550 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:55.632528    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/9503836b-029a-462a-bea1-ddc8684b2b65-signing-cabundle\") pod \"service-ca-758f599c5-vqfmq\" (UID: \"9503836b-029a-462a-bea1-ddc8684b2b65\") " pod="openshift-service-ca/service-ca-758f599c5-vqfmq"
Jul 07 17:08:55.634341 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:55.634321    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/9503836b-029a-462a-bea1-ddc8684b2b65-signing-key\") pod \"service-ca-758f599c5-vqfmq\" (UID: \"9503836b-029a-462a-bea1-ddc8684b2b65\") " pod="openshift-service-ca/service-ca-758f599c5-vqfmq"
Jul 07 17:08:55.644734 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:55.644712    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmp2q\" (UniqueName: \"kubernetes.io/projected/9503836b-029a-462a-bea1-ddc8684b2b65-kube-api-access-hmp2q\") pod \"service-ca-758f599c5-vqfmq\" (UID: \"9503836b-029a-462a-bea1-ddc8684b2b65\") " pod="openshift-service-ca/service-ca-758f599c5-vqfmq"
Jul 07 17:08:55.826043 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:55.826020    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-vqfmq"
Jul 07 17:08:55.950982 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:55.950956    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-vqfmq"]
Jul 07 17:08:55.953307 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:08:55.953282    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9503836b_029a_462a_bea1_ddc8684b2b65.slice/crio-dd568b9a258f39a7f7585611daa93169623fbd5c124c644be6780190ee8e3ff4 WatchSource:0}: Error finding container dd568b9a258f39a7f7585611daa93169623fbd5c124c644be6780190ee8e3ff4: Status 404 returned error can't find the container with id dd568b9a258f39a7f7585611daa93169623fbd5c124c644be6780190ee8e3ff4
Jul 07 17:08:56.361379 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:56.361304    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-vqfmq" event={"ID":"9503836b-029a-462a-bea1-ddc8684b2b65","Type":"ContainerStarted","Data":"fc751917730cffc7b1ca86e1dc4e2beac3fe8bed58396e6274fa7ab1c8e3f68f"}
Jul 07 17:08:56.361379 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:56.361355    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-vqfmq" event={"ID":"9503836b-029a-462a-bea1-ddc8684b2b65","Type":"ContainerStarted","Data":"dd568b9a258f39a7f7585611daa93169623fbd5c124c644be6780190ee8e3ff4"}
Jul 07 17:08:56.380844 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:56.380793    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-vqfmq" podStartSLOduration=1.380772852 podStartE2EDuration="1.380772852s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:08:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:08:56.379194568 +0000 UTC m=+59.875273078" watchObservedRunningTime="2026-07-07 17:08:56.380772852 +0000 UTC m=+59.876851342"
Jul 07 17:08:59.295184 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:59.295158    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vhsvz"
Jul 07 17:08:59.358447 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:59.358424    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/299d1c5f-8255-4e02-9cef-7a80ce5f88d7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ph5mc\" (UID: \"299d1c5f-8255-4e02-9cef-7a80ce5f88d7\") " pod="openshift-insights/insights-runtime-extractor-ph5mc"
Jul 07 17:08:59.358604 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:59.358548    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 17:08:59.358604 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:08:59.358595    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/299d1c5f-8255-4e02-9cef-7a80ce5f88d7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ph5mc\" (UID: \"299d1c5f-8255-4e02-9cef-7a80ce5f88d7\") " pod="openshift-insights/insights-runtime-extractor-ph5mc"
Jul 07 17:08:59.358740 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:59.358625    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/299d1c5f-8255-4e02-9cef-7a80ce5f88d7-kube-rbac-proxy-cm podName:299d1c5f-8255-4e02-9cef-7a80ce5f88d7 nodeName:}" failed. No retries permitted until 2026-07-07 17:09:07.358605971 +0000 UTC m=+70.854684471 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/299d1c5f-8255-4e02-9cef-7a80ce5f88d7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ph5mc" (UID: "299d1c5f-8255-4e02-9cef-7a80ce5f88d7") : configmap "kube-rbac-proxy" not found
Jul 07 17:08:59.358740 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:59.358666    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 17:08:59.358740 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:08:59.358706    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/299d1c5f-8255-4e02-9cef-7a80ce5f88d7-insights-runtime-extractor-tls podName:299d1c5f-8255-4e02-9cef-7a80ce5f88d7 nodeName:}" failed. No retries permitted until 2026-07-07 17:09:07.358691464 +0000 UTC m=+70.854769950 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/299d1c5f-8255-4e02-9cef-7a80ce5f88d7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ph5mc" (UID: "299d1c5f-8255-4e02-9cef-7a80ce5f88d7") : secret "insights-runtime-extractor-tls" not found
Jul 07 17:09:02.681871 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:02.681836    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c398fff8-ec63-4063-b3a4-92bf0aa5df9c-metrics-certs\") pod \"network-metrics-daemon-24rwk\" (UID: \"c398fff8-ec63-4063-b3a4-92bf0aa5df9c\") " pod="openshift-multus/network-metrics-daemon-24rwk"
Jul 07 17:09:02.684257 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:02.684240    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 07 17:09:02.692355 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:09:02.692337    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 07 17:09:02.692445 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:09:02.692414    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c398fff8-ec63-4063-b3a4-92bf0aa5df9c-metrics-certs podName:c398fff8-ec63-4063-b3a4-92bf0aa5df9c nodeName:}" failed. No retries permitted until 2026-07-07 17:10:06.692391845 +0000 UTC m=+130.188470352 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c398fff8-ec63-4063-b3a4-92bf0aa5df9c-metrics-certs") pod "network-metrics-daemon-24rwk" (UID: "c398fff8-ec63-4063-b3a4-92bf0aa5df9c") : secret "metrics-daemon-secret" not found
Jul 07 17:09:02.782452 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:02.782426    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbv8k\" (UniqueName: \"kubernetes.io/projected/d6844e3d-9b85-4d30-86db-35466ed9a36e-kube-api-access-jbv8k\") pod \"network-check-target-fm4tg\" (UID: \"d6844e3d-9b85-4d30-86db-35466ed9a36e\") " pod="openshift-network-diagnostics/network-check-target-fm4tg"
Jul 07 17:09:02.782553 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:02.782454    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5d706921-ddd7-4b39-840d-a03cef437d64-original-pull-secret\") pod \"global-pull-secret-syncer-wqh28\" (UID: \"5d706921-ddd7-4b39-840d-a03cef437d64\") " pod="kube-system/global-pull-secret-syncer-wqh28"
Jul 07 17:09:02.789065 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:02.789047    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 07 17:09:02.789931 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:02.789918    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 07 17:09:02.794853 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:02.794829    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5d706921-ddd7-4b39-840d-a03cef437d64-original-pull-secret\") pod \"global-pull-secret-syncer-wqh28\" (UID: \"5d706921-ddd7-4b39-840d-a03cef437d64\") " pod="kube-system/global-pull-secret-syncer-wqh28"
Jul 07 17:09:02.795195 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:02.795179    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 07 17:09:02.805080 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:02.805059    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbv8k\" (UniqueName: \"kubernetes.io/projected/d6844e3d-9b85-4d30-86db-35466ed9a36e-kube-api-access-jbv8k\") pod \"network-check-target-fm4tg\" (UID: \"d6844e3d-9b85-4d30-86db-35466ed9a36e\") " pod="openshift-network-diagnostics/network-check-target-fm4tg"
Jul 07 17:09:03.042858 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:03.042804    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jk68m\""
Jul 07 17:09:03.046347 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:03.046332    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqh28"
Jul 07 17:09:03.050462 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:03.050448    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fm4tg"
Jul 07 17:09:03.201538 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:03.201514    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wqh28"]
Jul 07 17:09:03.204149 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:09:03.204123    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d706921_ddd7_4b39_840d_a03cef437d64.slice/crio-c0f044588f8f6024357698f0cf775f44e8fcd3ffe2bd8827a4ba6b055793af6b WatchSource:0}: Error finding container c0f044588f8f6024357698f0cf775f44e8fcd3ffe2bd8827a4ba6b055793af6b: Status 404 returned error can't find the container with id c0f044588f8f6024357698f0cf775f44e8fcd3ffe2bd8827a4ba6b055793af6b
Jul 07 17:09:03.215747 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:03.215722    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fm4tg"]
Jul 07 17:09:03.218842 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:09:03.218821    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6844e3d_9b85_4d30_86db_35466ed9a36e.slice/crio-123a774e160e3cfeb6674363519e339ffbd31c232b83d98cdb10129e077d145b WatchSource:0}: Error finding container 123a774e160e3cfeb6674363519e339ffbd31c232b83d98cdb10129e077d145b: Status 404 returned error can't find the container with id 123a774e160e3cfeb6674363519e339ffbd31c232b83d98cdb10129e077d145b
Jul 07 17:09:03.379976 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:03.379909    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fm4tg" event={"ID":"d6844e3d-9b85-4d30-86db-35466ed9a36e","Type":"ContainerStarted","Data":"123a774e160e3cfeb6674363519e339ffbd31c232b83d98cdb10129e077d145b"}
Jul 07 17:09:03.380948 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:03.380925    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wqh28" event={"ID":"5d706921-ddd7-4b39-840d-a03cef437d64","Type":"ContainerStarted","Data":"c0f044588f8f6024357698f0cf775f44e8fcd3ffe2bd8827a4ba6b055793af6b"}
Jul 07 17:09:03.387251 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:03.387223    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-registry-tls\") pod \"image-registry-b4c4c8fd5-x4qmh\" (UID: \"9535133b-ad4d-42dd-a756-c4d4c1f1fdc5\") " pod="openshift-image-registry/image-registry-b4c4c8fd5-x4qmh"
Jul 07 17:09:03.389487 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:03.389471    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-registry-tls\") pod \"image-registry-b4c4c8fd5-x4qmh\" (UID: \"9535133b-ad4d-42dd-a756-c4d4c1f1fdc5\") " pod="openshift-image-registry/image-registry-b4c4c8fd5-x4qmh"
Jul 07 17:09:03.435356 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:03.435334    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-n59wn\""
Jul 07 17:09:03.441509 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:03.441493    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b4c4c8fd5-x4qmh"
Jul 07 17:09:03.572961 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:03.572932    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-b4c4c8fd5-x4qmh"]
Jul 07 17:09:03.575313 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:09:03.575286    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9535133b_ad4d_42dd_a756_c4d4c1f1fdc5.slice/crio-6e360e3765e9d5b6a4c0d865801659f0f60527929eb4c91d17efea57f9f6c203 WatchSource:0}: Error finding container 6e360e3765e9d5b6a4c0d865801659f0f60527929eb4c91d17efea57f9f6c203: Status 404 returned error can't find the container with id 6e360e3765e9d5b6a4c0d865801659f0f60527929eb4c91d17efea57f9f6c203
Jul 07 17:09:04.194003 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:04.193948    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b04e3abe-c8da-4123-ae78-3d978812545d-metrics-tls\") pod \"dns-default-nbkp6\" (UID: \"b04e3abe-c8da-4123-ae78-3d978812545d\") " pod="openshift-dns/dns-default-nbkp6"
Jul 07 17:09:04.205062 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:04.204999    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b04e3abe-c8da-4123-ae78-3d978812545d-metrics-tls\") pod \"dns-default-nbkp6\" (UID: \"b04e3abe-c8da-4123-ae78-3d978812545d\") " pod="openshift-dns/dns-default-nbkp6"
Jul 07 17:09:04.387027 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:04.386993    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b4c4c8fd5-x4qmh" event={"ID":"9535133b-ad4d-42dd-a756-c4d4c1f1fdc5","Type":"ContainerStarted","Data":"01caacf2a38ba1127cc178284143d53ec494e4691f27aec37c01925d253ca02b"}
Jul 07 17:09:04.387177 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:04.387047    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b4c4c8fd5-x4qmh" event={"ID":"9535133b-ad4d-42dd-a756-c4d4c1f1fdc5","Type":"ContainerStarted","Data":"6e360e3765e9d5b6a4c0d865801659f0f60527929eb4c91d17efea57f9f6c203"}
Jul 07 17:09:04.387252 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:04.387226    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-b4c4c8fd5-x4qmh"
Jul 07 17:09:04.412782 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:04.412735    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-b4c4c8fd5-x4qmh" podStartSLOduration=17.412716818 podStartE2EDuration="17.412716818s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:08:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:09:04.412375664 +0000 UTC m=+67.908454175" watchObservedRunningTime="2026-07-07 17:09:04.412716818 +0000 UTC m=+67.908795330"
Jul 07 17:09:04.450401 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:04.450341    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9r2xr\""
Jul 07 17:09:04.459363 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:04.459342    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nbkp6"
Jul 07 17:09:04.611881 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:04.611800    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nbkp6"]
Jul 07 17:09:04.613598 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:09:04.613568    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb04e3abe_c8da_4123_ae78_3d978812545d.slice/crio-4e81f7019eed78aade3fa649e4e11cda6d177dc9c70f71efac58cc789066bf72 WatchSource:0}: Error finding container 4e81f7019eed78aade3fa649e4e11cda6d177dc9c70f71efac58cc789066bf72: Status 404 returned error can't find the container with id 4e81f7019eed78aade3fa649e4e11cda6d177dc9c70f71efac58cc789066bf72
Jul 07 17:09:05.392534 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:05.392488    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nbkp6" event={"ID":"b04e3abe-c8da-4123-ae78-3d978812545d","Type":"ContainerStarted","Data":"4e81f7019eed78aade3fa649e4e11cda6d177dc9c70f71efac58cc789066bf72"}
Jul 07 17:09:07.421617 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:07.421582    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/299d1c5f-8255-4e02-9cef-7a80ce5f88d7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ph5mc\" (UID: \"299d1c5f-8255-4e02-9cef-7a80ce5f88d7\") " pod="openshift-insights/insights-runtime-extractor-ph5mc"
Jul 07 17:09:07.422039 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:07.421644    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/299d1c5f-8255-4e02-9cef-7a80ce5f88d7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ph5mc\" (UID: \"299d1c5f-8255-4e02-9cef-7a80ce5f88d7\") " pod="openshift-insights/insights-runtime-extractor-ph5mc"
Jul 07 17:09:07.422140 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:07.422116    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/299d1c5f-8255-4e02-9cef-7a80ce5f88d7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ph5mc\" (UID: \"299d1c5f-8255-4e02-9cef-7a80ce5f88d7\") " pod="openshift-insights/insights-runtime-extractor-ph5mc"
Jul 07 17:09:07.424259 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:07.424231    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/299d1c5f-8255-4e02-9cef-7a80ce5f88d7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ph5mc\" (UID: \"299d1c5f-8255-4e02-9cef-7a80ce5f88d7\") " pod="openshift-insights/insights-runtime-extractor-ph5mc"
Jul 07 17:09:07.528836 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:07.528811    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-kb2xx\""
Jul 07 17:09:07.536941 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:07.536916    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ph5mc"
Jul 07 17:09:09.408395 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:09.408360    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fm4tg" event={"ID":"d6844e3d-9b85-4d30-86db-35466ed9a36e","Type":"ContainerStarted","Data":"5f27f8128fd5660f903d98125f946bcac22eb442ccc6d121e3fb06fccaf8ef66"}
Jul 07 17:09:09.408737 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:09.408442    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-fm4tg"
Jul 07 17:09:09.471799 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:09.471746    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-fm4tg" podStartSLOduration=65.380332416 podStartE2EDuration="1m11.471724374s" totalImagesPullingTime="6.091391958s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:07:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:09:09.428501467 +0000 UTC m=+72.924579997" watchObservedRunningTime="2026-07-07 17:09:09.471724374 +0000 UTC m=+72.967802899"
Jul 07 17:09:09.472594 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:09.472569    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ph5mc"]
Jul 07 17:09:10.415229 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:10.415065    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ph5mc" event={"ID":"299d1c5f-8255-4e02-9cef-7a80ce5f88d7","Type":"ContainerStarted","Data":"6e06485500a6070b5fb03a2cf05b1c989fa8f9da1e1da468f5dab646354e7d8d"}
Jul 07 17:09:10.418359 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:10.417480    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nbkp6" event={"ID":"b04e3abe-c8da-4123-ae78-3d978812545d","Type":"ContainerStarted","Data":"f6fea3b1da1de5cca7a305d1506422844ae9a943b783579642ab8795d40391e5"}
Jul 07 17:09:10.418359 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:10.417536    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nbkp6" event={"ID":"b04e3abe-c8da-4123-ae78-3d978812545d","Type":"ContainerStarted","Data":"bbd12369463c4928a074182ed8d9fe0afadf6866cd10e363d91188afa4a65f24"}
Jul 07 17:09:10.418359 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:10.418288    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-nbkp6"
Jul 07 17:09:10.420576 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:10.420537    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wqh28" event={"ID":"5d706921-ddd7-4b39-840d-a03cef437d64","Type":"ContainerStarted","Data":"ea7917dad64e4680b6af315a6cf303d01ee217bffa0ed4d2194b5f7d564d35b7"}
Jul 07 17:09:10.436925 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:10.436447    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-nbkp6" podStartSLOduration=33.738390001 podStartE2EDuration="38.436431028s" totalImagesPullingTime="4.698041027s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:08:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:09:10.43592427 +0000 UTC m=+73.932002781" watchObservedRunningTime="2026-07-07 17:09:10.436431028 +0000 UTC m=+73.932509544"
Jul 07 17:09:13.430674 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:13.430642    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ph5mc" event={"ID":"299d1c5f-8255-4e02-9cef-7a80ce5f88d7","Type":"ContainerStarted","Data":"5fd86ea59c9a3e5137cfce464e141a3ec8a152ab644a3919d0ee26ea946e145c"}
Jul 07 17:09:14.436560 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:14.436526    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ph5mc" event={"ID":"299d1c5f-8255-4e02-9cef-7a80ce5f88d7","Type":"ContainerStarted","Data":"4c8bae1fbf5a42efbb3bf5f595dbd9f598dedd2276da36b7d93f7882da218739"}
Jul 07 17:09:17.449333 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:17.449230    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ph5mc" event={"ID":"299d1c5f-8255-4e02-9cef-7a80ce5f88d7","Type":"ContainerStarted","Data":"83f6d40bfc710cc28c56018635ed914f25e542d8f70bdc43705d1d4f02554801"}
Jul 07 17:09:17.469210 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:17.469153    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-wqh28" podStartSLOduration=73.347073934 podStartE2EDuration="1m19.46913813s" totalImagesPullingTime="6.122064196s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:07:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:09:10.453454785 +0000 UTC m=+73.949533295" watchObservedRunningTime="2026-07-07 17:09:17.46913813 +0000 UTC m=+80.965216638"
Jul 07 17:09:17.469397 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:17.469378    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-ph5mc" podStartSLOduration=19.670665617 podStartE2EDuration="26.469375095s" totalImagesPullingTime="6.798709478s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:08:51 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:09:17.468298246 +0000 UTC m=+80.964376754" watchObservedRunningTime="2026-07-07 17:09:17.469375095 +0000 UTC m=+80.965453602"
Jul 07 17:09:20.853280 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:20.853252    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9b447d5bd-pxvn5"]
Jul 07 17:09:20.857429 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:20.857411    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9b447d5bd-pxvn5"
Jul 07 17:09:20.862175 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:20.862155    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 07 17:09:20.862811 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:20.862792    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-kzmz5\""
Jul 07 17:09:20.862992 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:20.862971    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 07 17:09:20.863116 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:20.863089    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 07 17:09:20.863732 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:20.863715    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 07 17:09:20.884023 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:20.884000    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-b4c4c8fd5-x4qmh"]
Jul 07 17:09:20.885115 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:20.885093    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9b447d5bd-pxvn5"]
Jul 07 17:09:20.888954 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:20.888931    2579 patch_prober.go:28] interesting pod/image-registry-b4c4c8fd5-x4qmh container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 07 17:09:20.889045 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:20.888974    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-b4c4c8fd5-x4qmh" podUID="9535133b-ad4d-42dd-a756-c4d4c1f1fdc5" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 17:09:20.922594 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:20.922573    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d8fde104-5d8b-4d7f-9465-804e3d940dc5-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-9b447d5bd-pxvn5\" (UID: \"d8fde104-5d8b-4d7f-9465-804e3d940dc5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9b447d5bd-pxvn5"
Jul 07 17:09:20.922686 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:20.922608    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97mvp\" (UniqueName: \"kubernetes.io/projected/d8fde104-5d8b-4d7f-9465-804e3d940dc5-kube-api-access-97mvp\") pod \"managed-serviceaccount-addon-agent-9b447d5bd-pxvn5\" (UID: \"d8fde104-5d8b-4d7f-9465-804e3d940dc5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9b447d5bd-pxvn5"
Jul 07 17:09:20.990780 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:20.990755    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-74b85b9dfc-gpbwp"]
Jul 07 17:09:20.993605 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:20.993588    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74b85b9dfc-gpbwp"
Jul 07 17:09:21.023060 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:21.023036    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d8fde104-5d8b-4d7f-9465-804e3d940dc5-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-9b447d5bd-pxvn5\" (UID: \"d8fde104-5d8b-4d7f-9465-804e3d940dc5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9b447d5bd-pxvn5"
Jul 07 17:09:21.023157 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:21.023073    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97mvp\" (UniqueName: \"kubernetes.io/projected/d8fde104-5d8b-4d7f-9465-804e3d940dc5-kube-api-access-97mvp\") pod \"managed-serviceaccount-addon-agent-9b447d5bd-pxvn5\" (UID: \"d8fde104-5d8b-4d7f-9465-804e3d940dc5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9b447d5bd-pxvn5"
Jul 07 17:09:21.025714 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:21.025691    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d8fde104-5d8b-4d7f-9465-804e3d940dc5-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-9b447d5bd-pxvn5\" (UID: \"d8fde104-5d8b-4d7f-9465-804e3d940dc5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9b447d5bd-pxvn5"
Jul 07 17:09:21.036906 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:21.036873    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-74b85b9dfc-gpbwp"]
Jul 07 17:09:21.064176 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:21.064152    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-97mvp\" (UniqueName: \"kubernetes.io/projected/d8fde104-5d8b-4d7f-9465-804e3d940dc5-kube-api-access-97mvp\") pod \"managed-serviceaccount-addon-agent-9b447d5bd-pxvn5\" (UID: \"d8fde104-5d8b-4d7f-9465-804e3d940dc5\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9b447d5bd-pxvn5"
Jul 07 17:09:21.123961 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:21.123907    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7954a9ce-fc31-4880-9914-5d78f8a3bc1b-image-registry-private-configuration\") pod \"image-registry-74b85b9dfc-gpbwp\" (UID: \"7954a9ce-fc31-4880-9914-5d78f8a3bc1b\") " pod="openshift-image-registry/image-registry-74b85b9dfc-gpbwp"
Jul 07 17:09:21.123961 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:21.123938    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7954a9ce-fc31-4880-9914-5d78f8a3bc1b-installation-pull-secrets\") pod \"image-registry-74b85b9dfc-gpbwp\" (UID: \"7954a9ce-fc31-4880-9914-5d78f8a3bc1b\") " pod="openshift-image-registry/image-registry-74b85b9dfc-gpbwp"
Jul 07 17:09:21.124074 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:21.123981    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7954a9ce-fc31-4880-9914-5d78f8a3bc1b-ca-trust-extracted\") pod \"image-registry-74b85b9dfc-gpbwp\" (UID: \"7954a9ce-fc31-4880-9914-5d78f8a3bc1b\") " pod="openshift-image-registry/image-registry-74b85b9dfc-gpbwp"
Jul 07 17:09:21.124074 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:21.124000    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7954a9ce-fc31-4880-9914-5d78f8a3bc1b-bound-sa-token\") pod \"image-registry-74b85b9dfc-gpbwp\" (UID: \"7954a9ce-fc31-4880-9914-5d78f8a3bc1b\") " pod="openshift-image-registry/image-registry-74b85b9dfc-gpbwp"
Jul 07 17:09:21.124074 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:21.124016    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgksl\" (UniqueName: \"kubernetes.io/projected/7954a9ce-fc31-4880-9914-5d78f8a3bc1b-kube-api-access-tgksl\") pod \"image-registry-74b85b9dfc-gpbwp\" (UID: \"7954a9ce-fc31-4880-9914-5d78f8a3bc1b\") " pod="openshift-image-registry/image-registry-74b85b9dfc-gpbwp"
Jul 07 17:09:21.124074 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:21.124051    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7954a9ce-fc31-4880-9914-5d78f8a3bc1b-registry-certificates\") pod \"image-registry-74b85b9dfc-gpbwp\" (UID: \"7954a9ce-fc31-4880-9914-5d78f8a3bc1b\") " pod="openshift-image-registry/image-registry-74b85b9dfc-gpbwp"
Jul 07 17:09:21.124191 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:21.124113    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7954a9ce-fc31-4880-9914-5d78f8a3bc1b-registry-tls\") pod \"image-registry-74b85b9dfc-gpbwp\" (UID: \"7954a9ce-fc31-4880-9914-5d78f8a3bc1b\") " pod="openshift-image-registry/image-registry-74b85b9dfc-gpbwp"
Jul 07 17:09:21.124191 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:21.124136    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7954a9ce-fc31-4880-9914-5d78f8a3bc1b-trusted-ca\") pod \"image-registry-74b85b9dfc-gpbwp\" (UID: \"7954a9ce-fc31-4880-9914-5d78f8a3bc1b\") " pod="openshift-image-registry/image-registry-74b85b9dfc-gpbwp"
Jul 07 17:09:21.177558 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:21.177532    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9b447d5bd-pxvn5"
Jul 07 17:09:21.225744 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:21.224464    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7954a9ce-fc31-4880-9914-5d78f8a3bc1b-registry-certificates\") pod \"image-registry-74b85b9dfc-gpbwp\" (UID: \"7954a9ce-fc31-4880-9914-5d78f8a3bc1b\") " pod="openshift-image-registry/image-registry-74b85b9dfc-gpbwp"
Jul 07 17:09:21.225744 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:21.224531    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7954a9ce-fc31-4880-9914-5d78f8a3bc1b-registry-tls\") pod \"image-registry-74b85b9dfc-gpbwp\" (UID: \"7954a9ce-fc31-4880-9914-5d78f8a3bc1b\") " pod="openshift-image-registry/image-registry-74b85b9dfc-gpbwp"
Jul 07 17:09:21.225744 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:21.224557    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7954a9ce-fc31-4880-9914-5d78f8a3bc1b-trusted-ca\") pod \"image-registry-74b85b9dfc-gpbwp\" (UID: \"7954a9ce-fc31-4880-9914-5d78f8a3bc1b\") " pod="openshift-image-registry/image-registry-74b85b9dfc-gpbwp"
Jul 07 17:09:21.225744 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:21.224604    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7954a9ce-fc31-4880-9914-5d78f8a3bc1b-image-registry-private-configuration\") pod \"image-registry-74b85b9dfc-gpbwp\" (UID: \"7954a9ce-fc31-4880-9914-5d78f8a3bc1b\") " pod="openshift-image-registry/image-registry-74b85b9dfc-gpbwp"
Jul 07 17:09:21.225744 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:21.224631    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7954a9ce-fc31-4880-9914-5d78f8a3bc1b-installation-pull-secrets\") pod \"image-registry-74b85b9dfc-gpbwp\" (UID: \"7954a9ce-fc31-4880-9914-5d78f8a3bc1b\") " pod="openshift-image-registry/image-registry-74b85b9dfc-gpbwp"
Jul 07 17:09:21.225744 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:21.224675    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7954a9ce-fc31-4880-9914-5d78f8a3bc1b-ca-trust-extracted\") pod \"image-registry-74b85b9dfc-gpbwp\" (UID: \"7954a9ce-fc31-4880-9914-5d78f8a3bc1b\") " pod="openshift-image-registry/image-registry-74b85b9dfc-gpbwp"
Jul 07 17:09:21.225744 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:21.224707    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7954a9ce-fc31-4880-9914-5d78f8a3bc1b-bound-sa-token\") pod \"image-registry-74b85b9dfc-gpbwp\" (UID: \"7954a9ce-fc31-4880-9914-5d78f8a3bc1b\") " pod="openshift-image-registry/image-registry-74b85b9dfc-gpbwp"
Jul 07 17:09:21.225744 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:21.224732    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgksl\" (UniqueName: \"kubernetes.io/projected/7954a9ce-fc31-4880-9914-5d78f8a3bc1b-kube-api-access-tgksl\") pod \"image-registry-74b85b9dfc-gpbwp\" (UID: \"7954a9ce-fc31-4880-9914-5d78f8a3bc1b\") " pod="openshift-image-registry/image-registry-74b85b9dfc-gpbwp"
Jul 07 17:09:21.225744 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:21.225410    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7954a9ce-fc31-4880-9914-5d78f8a3bc1b-registry-certificates\") pod \"image-registry-74b85b9dfc-gpbwp\" (UID: \"7954a9ce-fc31-4880-9914-5d78f8a3bc1b\") " pod="openshift-image-registry/image-registry-74b85b9dfc-gpbwp"
Jul 07 17:09:21.226373 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:21.225807    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7954a9ce-fc31-4880-9914-5d78f8a3bc1b-ca-trust-extracted\") pod \"image-registry-74b85b9dfc-gpbwp\" (UID: \"7954a9ce-fc31-4880-9914-5d78f8a3bc1b\") " pod="openshift-image-registry/image-registry-74b85b9dfc-gpbwp"
Jul 07 17:09:21.227312 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:21.226711    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7954a9ce-fc31-4880-9914-5d78f8a3bc1b-trusted-ca\") pod \"image-registry-74b85b9dfc-gpbwp\" (UID: \"7954a9ce-fc31-4880-9914-5d78f8a3bc1b\") " pod="openshift-image-registry/image-registry-74b85b9dfc-gpbwp"
Jul 07 17:09:21.228708 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:21.228615    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7954a9ce-fc31-4880-9914-5d78f8a3bc1b-registry-tls\") pod \"image-registry-74b85b9dfc-gpbwp\" (UID: \"7954a9ce-fc31-4880-9914-5d78f8a3bc1b\") " pod="openshift-image-registry/image-registry-74b85b9dfc-gpbwp"
Jul 07 17:09:21.229179 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:21.229158    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7954a9ce-fc31-4880-9914-5d78f8a3bc1b-image-registry-private-configuration\") pod \"image-registry-74b85b9dfc-gpbwp\" (UID: \"7954a9ce-fc31-4880-9914-5d78f8a3bc1b\") " pod="openshift-image-registry/image-registry-74b85b9dfc-gpbwp"
Jul 07 17:09:21.229590 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:21.229573    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7954a9ce-fc31-4880-9914-5d78f8a3bc1b-installation-pull-secrets\") pod \"image-registry-74b85b9dfc-gpbwp\" (UID: \"7954a9ce-fc31-4880-9914-5d78f8a3bc1b\") " pod="openshift-image-registry/image-registry-74b85b9dfc-gpbwp"
Jul 07 17:09:21.240071 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:21.240045    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7954a9ce-fc31-4880-9914-5d78f8a3bc1b-bound-sa-token\") pod \"image-registry-74b85b9dfc-gpbwp\" (UID: \"7954a9ce-fc31-4880-9914-5d78f8a3bc1b\") " pod="openshift-image-registry/image-registry-74b85b9dfc-gpbwp"
Jul 07 17:09:21.249676 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:21.249640    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgksl\" (UniqueName: \"kubernetes.io/projected/7954a9ce-fc31-4880-9914-5d78f8a3bc1b-kube-api-access-tgksl\") pod \"image-registry-74b85b9dfc-gpbwp\" (UID: \"7954a9ce-fc31-4880-9914-5d78f8a3bc1b\") " pod="openshift-image-registry/image-registry-74b85b9dfc-gpbwp"
Jul 07 17:09:21.301051 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:21.301026    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74b85b9dfc-gpbwp"
Jul 07 17:09:21.309672 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:21.309649    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9b447d5bd-pxvn5"]
Jul 07 17:09:21.312239 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:09:21.312216    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd8fde104_5d8b_4d7f_9465_804e3d940dc5.slice/crio-4b62c63e3dce9c48c75ef09c8ab94183ad1c029bc38af70e0b6ab7a363397a82 WatchSource:0}: Error finding container 4b62c63e3dce9c48c75ef09c8ab94183ad1c029bc38af70e0b6ab7a363397a82: Status 404 returned error can't find the container with id 4b62c63e3dce9c48c75ef09c8ab94183ad1c029bc38af70e0b6ab7a363397a82
Jul 07 17:09:21.428698 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:21.428673    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-nbkp6"
Jul 07 17:09:21.436107 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:21.436081    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-74b85b9dfc-gpbwp"]
Jul 07 17:09:21.439322 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:09:21.439301    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7954a9ce_fc31_4880_9914_5d78f8a3bc1b.slice/crio-cd994e1ebf72ced35f1e2f04a70b811bc009eecb6fe2169075f67be6d28b0d7a WatchSource:0}: Error finding container cd994e1ebf72ced35f1e2f04a70b811bc009eecb6fe2169075f67be6d28b0d7a: Status 404 returned error can't find the container with id cd994e1ebf72ced35f1e2f04a70b811bc009eecb6fe2169075f67be6d28b0d7a
Jul 07 17:09:21.465303 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:21.465276    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74b85b9dfc-gpbwp" event={"ID":"7954a9ce-fc31-4880-9914-5d78f8a3bc1b","Type":"ContainerStarted","Data":"cd994e1ebf72ced35f1e2f04a70b811bc009eecb6fe2169075f67be6d28b0d7a"}
Jul 07 17:09:21.466298 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:21.466280    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9b447d5bd-pxvn5" event={"ID":"d8fde104-5d8b-4d7f-9465-804e3d940dc5","Type":"ContainerStarted","Data":"4b62c63e3dce9c48c75ef09c8ab94183ad1c029bc38af70e0b6ab7a363397a82"}
Jul 07 17:09:22.472485 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:22.472448    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74b85b9dfc-gpbwp" event={"ID":"7954a9ce-fc31-4880-9914-5d78f8a3bc1b","Type":"ContainerStarted","Data":"cfbb443f4da653fe31ee73cef463ac4ef57e91196eeb1b70e11721a68361d5e6"}
Jul 07 17:09:22.472954 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:22.472658    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-74b85b9dfc-gpbwp"
Jul 07 17:09:22.495101 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:22.495047    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-74b85b9dfc-gpbwp" podStartSLOduration=2.495027768 podStartE2EDuration="2.495027768s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:09:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:09:22.492786062 +0000 UTC m=+85.988864569" watchObservedRunningTime="2026-07-07 17:09:22.495027768 +0000 UTC m=+85.991106275"
Jul 07 17:09:24.480513 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:24.480481    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9b447d5bd-pxvn5" event={"ID":"d8fde104-5d8b-4d7f-9465-804e3d940dc5","Type":"ContainerStarted","Data":"ddf2901bfbd950570ef9c6aad7ed1e79a66d1febf674e34e723159a494971d77"}
Jul 07 17:09:24.497106 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:24.497062    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9b447d5bd-pxvn5" podStartSLOduration=1.7955234039999999 podStartE2EDuration="4.497045546s" totalImagesPullingTime="2.701522142s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:09:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:09:24.496182979 +0000 UTC m=+87.992261476" watchObservedRunningTime="2026-07-07 17:09:24.497045546 +0000 UTC m=+87.993124061"
Jul 07 17:09:27.176077 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:27.176046    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-x68w5"]
Jul 07 17:09:27.180546 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:27.180530    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x68w5"
Jul 07 17:09:27.187104 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:27.187080    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-8scrr\""
Jul 07 17:09:27.187209 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:27.187112    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 07 17:09:27.188044 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:27.187943    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 07 17:09:27.188310 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:27.188251    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 07 17:09:27.188387 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:27.188258    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 07 17:09:27.188387 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:27.188299    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 07 17:09:27.193981 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:27.193961    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-x68w5"]
Jul 07 17:09:27.266361 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:27.266338    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35ad1132-a443-447a-bee9-3fe909270852-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-x68w5\" (UID: \"35ad1132-a443-447a-bee9-3fe909270852\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x68w5"
Jul 07 17:09:27.266472 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:27.266367    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljnr8\" (UniqueName: \"kubernetes.io/projected/35ad1132-a443-447a-bee9-3fe909270852-kube-api-access-ljnr8\") pod \"prometheus-operator-567f7b8b7c-x68w5\" (UID: \"35ad1132-a443-447a-bee9-3fe909270852\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x68w5"
Jul 07 17:09:27.266472 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:27.266394    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/35ad1132-a443-447a-bee9-3fe909270852-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-x68w5\" (UID: \"35ad1132-a443-447a-bee9-3fe909270852\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x68w5"
Jul 07 17:09:27.266472 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:27.266419    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/35ad1132-a443-447a-bee9-3fe909270852-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-x68w5\" (UID: \"35ad1132-a443-447a-bee9-3fe909270852\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x68w5"
Jul 07 17:09:27.366919 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:27.366875    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/35ad1132-a443-447a-bee9-3fe909270852-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-x68w5\" (UID: \"35ad1132-a443-447a-bee9-3fe909270852\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x68w5"
Jul 07 17:09:27.367005 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:27.366943    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35ad1132-a443-447a-bee9-3fe909270852-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-x68w5\" (UID: \"35ad1132-a443-447a-bee9-3fe909270852\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x68w5"
Jul 07 17:09:27.367005 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:27.366963    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljnr8\" (UniqueName: \"kubernetes.io/projected/35ad1132-a443-447a-bee9-3fe909270852-kube-api-access-ljnr8\") pod \"prometheus-operator-567f7b8b7c-x68w5\" (UID: \"35ad1132-a443-447a-bee9-3fe909270852\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x68w5"
Jul 07 17:09:27.367005 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:27.366987    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/35ad1132-a443-447a-bee9-3fe909270852-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-x68w5\" (UID: \"35ad1132-a443-447a-bee9-3fe909270852\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x68w5"
Jul 07 17:09:27.367577 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:27.367557    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35ad1132-a443-447a-bee9-3fe909270852-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-x68w5\" (UID: \"35ad1132-a443-447a-bee9-3fe909270852\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x68w5"
Jul 07 17:09:27.369122 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:27.369101    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/35ad1132-a443-447a-bee9-3fe909270852-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-x68w5\" (UID: \"35ad1132-a443-447a-bee9-3fe909270852\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x68w5"
Jul 07 17:09:27.369197 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:27.369102    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/35ad1132-a443-447a-bee9-3fe909270852-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-x68w5\" (UID: \"35ad1132-a443-447a-bee9-3fe909270852\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x68w5"
Jul 07 17:09:27.376722 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:27.376705    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljnr8\" (UniqueName: \"kubernetes.io/projected/35ad1132-a443-447a-bee9-3fe909270852-kube-api-access-ljnr8\") pod \"prometheus-operator-567f7b8b7c-x68w5\" (UID: \"35ad1132-a443-447a-bee9-3fe909270852\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x68w5"
Jul 07 17:09:27.490972 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:27.490928    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x68w5"
Jul 07 17:09:27.616563 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:27.616532    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-x68w5"]
Jul 07 17:09:27.620014 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:09:27.619991    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35ad1132_a443_447a_bee9_3fe909270852.slice/crio-3986bac648acee10888917392879468c094c8ade646d5651bd6c30ae96aafb49 WatchSource:0}: Error finding container 3986bac648acee10888917392879468c094c8ade646d5651bd6c30ae96aafb49: Status 404 returned error can't find the container with id 3986bac648acee10888917392879468c094c8ade646d5651bd6c30ae96aafb49
Jul 07 17:09:28.496153 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:28.496114    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x68w5" event={"ID":"35ad1132-a443-447a-bee9-3fe909270852","Type":"ContainerStarted","Data":"3986bac648acee10888917392879468c094c8ade646d5651bd6c30ae96aafb49"}
Jul 07 17:09:29.502860 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:29.502775    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x68w5" event={"ID":"35ad1132-a443-447a-bee9-3fe909270852","Type":"ContainerStarted","Data":"ed06573501edeffd452da67802e7c0351dfeb2be5900b3dd7635a1f701fed4bc"}
Jul 07 17:09:29.502860 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:29.502835    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x68w5" event={"ID":"35ad1132-a443-447a-bee9-3fe909270852","Type":"ContainerStarted","Data":"97cc1e11e5e11e9e9ee58d9eaa3ac73f66c95cdd4b55e229c3df703a4cf08bd5"}
Jul 07 17:09:29.529576 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:29.529527    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-x68w5" podStartSLOduration=0.897425748 podStartE2EDuration="2.529508619s" totalImagesPullingTime="1.632082871s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:09:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:09:29.528654961 +0000 UTC m=+93.024733468" watchObservedRunningTime="2026-07-07 17:09:29.529508619 +0000 UTC m=+93.025587129"
Jul 07 17:09:30.888797 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:30.888763    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-b4c4c8fd5-x4qmh"
Jul 07 17:09:31.569404 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:31.569372    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-lx7tn"]
Jul 07 17:09:31.572810 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:31.572791    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lx7tn"
Jul 07 17:09:31.575871 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:31.575852    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 07 17:09:31.575989 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:31.575859    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 07 17:09:31.576144 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:31.576129    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 07 17:09:31.576335 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:31.576319    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-vvz5v\""
Jul 07 17:09:31.695087 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:31.695058    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764-node-exporter-tls\") pod \"node-exporter-lx7tn\" (UID: \"4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764\") " pod="openshift-monitoring/node-exporter-lx7tn"
Jul 07 17:09:31.695209 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:31.695107    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764-sys\") pod \"node-exporter-lx7tn\" (UID: \"4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764\") " pod="openshift-monitoring/node-exporter-lx7tn"
Jul 07 17:09:31.695209 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:31.695152    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zf5sl\" (UniqueName: \"kubernetes.io/projected/4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764-kube-api-access-zf5sl\") pod \"node-exporter-lx7tn\" (UID: \"4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764\") " pod="openshift-monitoring/node-exporter-lx7tn"
Jul 07 17:09:31.695209 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:31.695175    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764-root\") pod \"node-exporter-lx7tn\" (UID: \"4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764\") " pod="openshift-monitoring/node-exporter-lx7tn"
Jul 07 17:09:31.695209 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:31.695198    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764-node-exporter-accelerators-collector-config\") pod \"node-exporter-lx7tn\" (UID: \"4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764\") " pod="openshift-monitoring/node-exporter-lx7tn"
Jul 07 17:09:31.695366 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:31.695223    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764-metrics-client-ca\") pod \"node-exporter-lx7tn\" (UID: \"4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764\") " pod="openshift-monitoring/node-exporter-lx7tn"
Jul 07 17:09:31.695366 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:31.695249    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lx7tn\" (UID: \"4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764\") " pod="openshift-monitoring/node-exporter-lx7tn"
Jul 07 17:09:31.695366 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:31.695273    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764-node-exporter-textfile\") pod \"node-exporter-lx7tn\" (UID: \"4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764\") " pod="openshift-monitoring/node-exporter-lx7tn"
Jul 07 17:09:31.695452 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:31.695358    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764-node-exporter-wtmp\") pod \"node-exporter-lx7tn\" (UID: \"4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764\") " pod="openshift-monitoring/node-exporter-lx7tn"
Jul 07 17:09:31.796060 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:31.796033    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764-node-exporter-wtmp\") pod \"node-exporter-lx7tn\" (UID: \"4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764\") " pod="openshift-monitoring/node-exporter-lx7tn"
Jul 07 17:09:31.796060 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:31.796062    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764-node-exporter-tls\") pod \"node-exporter-lx7tn\" (UID: \"4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764\") " pod="openshift-monitoring/node-exporter-lx7tn"
Jul 07 17:09:31.796235 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:31.796085    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764-sys\") pod \"node-exporter-lx7tn\" (UID: \"4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764\") " pod="openshift-monitoring/node-exporter-lx7tn"
Jul 07 17:09:31.796235 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:31.796149    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764-sys\") pod \"node-exporter-lx7tn\" (UID: \"4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764\") " pod="openshift-monitoring/node-exporter-lx7tn"
Jul 07 17:09:31.796235 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:31.796177    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zf5sl\" (UniqueName: \"kubernetes.io/projected/4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764-kube-api-access-zf5sl\") pod \"node-exporter-lx7tn\" (UID: \"4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764\") " pod="openshift-monitoring/node-exporter-lx7tn"
Jul 07 17:09:31.796235 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:31.796214    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764-root\") pod \"node-exporter-lx7tn\" (UID: \"4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764\") " pod="openshift-monitoring/node-exporter-lx7tn"
Jul 07 17:09:31.796235 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:31.796224    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764-node-exporter-wtmp\") pod \"node-exporter-lx7tn\" (UID: \"4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764\") " pod="openshift-monitoring/node-exporter-lx7tn"
Jul 07 17:09:31.796467 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:31.796241    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764-node-exporter-accelerators-collector-config\") pod \"node-exporter-lx7tn\" (UID: \"4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764\") " pod="openshift-monitoring/node-exporter-lx7tn"
Jul 07 17:09:31.796467 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:31.796279    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764-root\") pod \"node-exporter-lx7tn\" (UID: \"4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764\") " pod="openshift-monitoring/node-exporter-lx7tn"
Jul 07 17:09:31.796467 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:31.796284    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764-metrics-client-ca\") pod \"node-exporter-lx7tn\" (UID: \"4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764\") " pod="openshift-monitoring/node-exporter-lx7tn"
Jul 07 17:09:31.796467 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:31.796324    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lx7tn\" (UID: \"4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764\") " pod="openshift-monitoring/node-exporter-lx7tn"
Jul 07 17:09:31.796467 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:31.796352    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764-node-exporter-textfile\") pod \"node-exporter-lx7tn\" (UID: \"4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764\") " pod="openshift-monitoring/node-exporter-lx7tn"
Jul 07 17:09:31.796729 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:31.796710    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764-node-exporter-textfile\") pod \"node-exporter-lx7tn\" (UID: \"4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764\") " pod="openshift-monitoring/node-exporter-lx7tn"
Jul 07 17:09:31.796979 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:31.796952    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764-node-exporter-accelerators-collector-config\") pod \"node-exporter-lx7tn\" (UID: \"4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764\") " pod="openshift-monitoring/node-exporter-lx7tn"
Jul 07 17:09:31.797616 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:31.797593    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764-metrics-client-ca\") pod \"node-exporter-lx7tn\" (UID: \"4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764\") " pod="openshift-monitoring/node-exporter-lx7tn"
Jul 07 17:09:31.798583 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:31.798563    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764-node-exporter-tls\") pod \"node-exporter-lx7tn\" (UID: \"4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764\") " pod="openshift-monitoring/node-exporter-lx7tn"
Jul 07 17:09:31.798809 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:31.798791    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lx7tn\" (UID: \"4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764\") " pod="openshift-monitoring/node-exporter-lx7tn"
Jul 07 17:09:31.812004 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:31.811984    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zf5sl\" (UniqueName: \"kubernetes.io/projected/4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764-kube-api-access-zf5sl\") pod \"node-exporter-lx7tn\" (UID: \"4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764\") " pod="openshift-monitoring/node-exporter-lx7tn"
Jul 07 17:09:31.882371 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:31.882318    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lx7tn"
Jul 07 17:09:31.898534 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:09:31.898512    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c4e36aa_1cb3_4b5b_bf45_3f05d4cc4764.slice/crio-e87ac1ae39aa57d1b851685ff54f5fcdc158ea0747d46a2b810f31c1fe428cdb WatchSource:0}: Error finding container e87ac1ae39aa57d1b851685ff54f5fcdc158ea0747d46a2b810f31c1fe428cdb: Status 404 returned error can't find the container with id e87ac1ae39aa57d1b851685ff54f5fcdc158ea0747d46a2b810f31c1fe428cdb
Jul 07 17:09:32.515045 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.515007    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lx7tn" event={"ID":"4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764","Type":"ContainerStarted","Data":"e87ac1ae39aa57d1b851685ff54f5fcdc158ea0747d46a2b810f31c1fe428cdb"}
Jul 07 17:09:32.653997 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.653952    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 07 17:09:32.660871 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.660836    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:09:32.663368 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.663351    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 07 17:09:32.663600 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.663584    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 07 17:09:32.663719 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.663677    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 07 17:09:32.663719 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.663692    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 07 17:09:32.663832 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.663692    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 07 17:09:32.663832 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.663798    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 07 17:09:32.663832 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.663818    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 07 17:09:32.664045 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.664024    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-df5fn\""
Jul 07 17:09:32.664161 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.664084    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 07 17:09:32.664161 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.664100    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 07 17:09:32.673497 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.673475    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 07 17:09:32.804249 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.804180    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ace5af06-9cac-4a3c-826a-c4a33dcbed95-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:09:32.804249 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.804219    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ace5af06-9cac-4a3c-826a-c4a33dcbed95-config-out\") pod \"alertmanager-main-0\" (UID: \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:09:32.804451 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.804251    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ace5af06-9cac-4a3c-826a-c4a33dcbed95-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:09:32.804451 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.804273    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vb5bm\" (UniqueName: \"kubernetes.io/projected/ace5af06-9cac-4a3c-826a-c4a33dcbed95-kube-api-access-vb5bm\") pod \"alertmanager-main-0\" (UID: \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:09:32.804451 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.804315    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ace5af06-9cac-4a3c-826a-c4a33dcbed95-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:09:32.804451 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.804334    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ace5af06-9cac-4a3c-826a-c4a33dcbed95-web-config\") pod \"alertmanager-main-0\" (UID: \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:09:32.804451 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.804364    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ace5af06-9cac-4a3c-826a-c4a33dcbed95-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:09:32.804451 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.804391    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ace5af06-9cac-4a3c-826a-c4a33dcbed95-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:09:32.804737 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.804508    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ace5af06-9cac-4a3c-826a-c4a33dcbed95-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:09:32.804737 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.804554    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ace5af06-9cac-4a3c-826a-c4a33dcbed95-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:09:32.804737 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.804587    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ace5af06-9cac-4a3c-826a-c4a33dcbed95-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:09:32.804737 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.804634    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ace5af06-9cac-4a3c-826a-c4a33dcbed95-config-volume\") pod \"alertmanager-main-0\" (UID: \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:09:32.804737 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.804677    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ace5af06-9cac-4a3c-826a-c4a33dcbed95-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:09:32.905462 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.905278    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ace5af06-9cac-4a3c-826a-c4a33dcbed95-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:09:32.905462 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.905325    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ace5af06-9cac-4a3c-826a-c4a33dcbed95-web-config\") pod \"alertmanager-main-0\" (UID: \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:09:32.905462 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.905448    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ace5af06-9cac-4a3c-826a-c4a33dcbed95-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:09:32.905992 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.905496    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ace5af06-9cac-4a3c-826a-c4a33dcbed95-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:09:32.905992 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.905547    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ace5af06-9cac-4a3c-826a-c4a33dcbed95-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:09:32.905992 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.905588    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ace5af06-9cac-4a3c-826a-c4a33dcbed95-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:09:32.905992 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.905620    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ace5af06-9cac-4a3c-826a-c4a33dcbed95-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:09:32.905992 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.905654    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ace5af06-9cac-4a3c-826a-c4a33dcbed95-config-volume\") pod \"alertmanager-main-0\" (UID: \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:09:32.905992 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.905681    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ace5af06-9cac-4a3c-826a-c4a33dcbed95-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:09:32.905992 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.905718    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ace5af06-9cac-4a3c-826a-c4a33dcbed95-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:09:32.905992 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.905744    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ace5af06-9cac-4a3c-826a-c4a33dcbed95-config-out\") pod \"alertmanager-main-0\" (UID: \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:09:32.905992 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.905773    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ace5af06-9cac-4a3c-826a-c4a33dcbed95-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:09:32.905992 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.905795    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vb5bm\" (UniqueName: \"kubernetes.io/projected/ace5af06-9cac-4a3c-826a-c4a33dcbed95-kube-api-access-vb5bm\") pod \"alertmanager-main-0\" (UID: \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:09:32.906438 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.906030    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ace5af06-9cac-4a3c-826a-c4a33dcbed95-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:09:32.907035 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.907013    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ace5af06-9cac-4a3c-826a-c4a33dcbed95-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:09:32.907179 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.907121    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ace5af06-9cac-4a3c-826a-c4a33dcbed95-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:09:32.909743 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.909686    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ace5af06-9cac-4a3c-826a-c4a33dcbed95-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:09:32.910393 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.910367    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ace5af06-9cac-4a3c-826a-c4a33dcbed95-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:09:32.910487 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.910372    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ace5af06-9cac-4a3c-826a-c4a33dcbed95-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:09:32.911228 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.911185    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ace5af06-9cac-4a3c-826a-c4a33dcbed95-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:09:32.911228 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.911197    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ace5af06-9cac-4a3c-826a-c4a33dcbed95-config-volume\") pod \"alertmanager-main-0\" (UID: \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:09:32.912030 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.912008    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ace5af06-9cac-4a3c-826a-c4a33dcbed95-web-config\") pod \"alertmanager-main-0\" (UID: \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:09:32.912549 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.912528    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ace5af06-9cac-4a3c-826a-c4a33dcbed95-config-out\") pod \"alertmanager-main-0\" (UID: \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:09:32.913210 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.912835    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ace5af06-9cac-4a3c-826a-c4a33dcbed95-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:09:32.913362 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.913325    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ace5af06-9cac-4a3c-826a-c4a33dcbed95-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:09:32.927872 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.927826    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vb5bm\" (UniqueName: \"kubernetes.io/projected/ace5af06-9cac-4a3c-826a-c4a33dcbed95-kube-api-access-vb5bm\") pod \"alertmanager-main-0\" (UID: \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:09:32.971629 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:32.971597    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:09:33.129822 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:33.129726    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 07 17:09:33.131688 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:09:33.131663    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podace5af06_9cac_4a3c_826a_c4a33dcbed95.slice/crio-0a1113aacfbf15aff401c3588181c6857867416f061ef8b799085d0382eb1a82 WatchSource:0}: Error finding container 0a1113aacfbf15aff401c3588181c6857867416f061ef8b799085d0382eb1a82: Status 404 returned error can't find the container with id 0a1113aacfbf15aff401c3588181c6857867416f061ef8b799085d0382eb1a82
Jul 07 17:09:33.519536 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:33.519505    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ace5af06-9cac-4a3c-826a-c4a33dcbed95","Type":"ContainerStarted","Data":"0a1113aacfbf15aff401c3588181c6857867416f061ef8b799085d0382eb1a82"}
Jul 07 17:09:33.520970 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:33.520945    2579 generic.go:356] "Generic (PLEG): container finished" podID="4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764" containerID="201c6d74e942a10cfa9608d86afad86c5858e218b62732ee27afd7d1dfb8f5a8" exitCode=0
Jul 07 17:09:33.521088 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:33.520982    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lx7tn" event={"ID":"4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764","Type":"ContainerDied","Data":"201c6d74e942a10cfa9608d86afad86c5858e218b62732ee27afd7d1dfb8f5a8"}
Jul 07 17:09:34.536828 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:34.536797    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lx7tn" event={"ID":"4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764","Type":"ContainerStarted","Data":"1923b84bca1b63efc78a8fc57343dc16055c198444dce6dcddae1b56a972ea02"}
Jul 07 17:09:34.537129 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:34.536907    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lx7tn" event={"ID":"4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764","Type":"ContainerStarted","Data":"3cccf8aa4081503785bb2f9bd11ba9989fd5c29efa784b8c04f1ac2daa138fb7"}
Jul 07 17:09:34.572447 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:34.572398    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-lx7tn" podStartSLOduration=2.368151937 podStartE2EDuration="3.572383319s" totalImagesPullingTime="1.133801885s" totalInitContainerRuntime="70.429497ms" isStatefulPod=true podCreationTimestamp="2026-07-07 17:09:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:09:34.570243005 +0000 UTC m=+98.066321512" watchObservedRunningTime="2026-07-07 17:09:34.572383319 +0000 UTC m=+98.068461846"
Jul 07 17:09:35.541823 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:35.541781    2579 generic.go:356] "Generic (PLEG): container finished" podID="ace5af06-9cac-4a3c-826a-c4a33dcbed95" containerID="e3f085ed8401f7f5efbed6861d5b8fd8149cf59ca5c4a83a988adccf7b2897f2" exitCode=0
Jul 07 17:09:35.542231 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:35.541881    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ace5af06-9cac-4a3c-826a-c4a33dcbed95","Type":"ContainerDied","Data":"e3f085ed8401f7f5efbed6861d5b8fd8149cf59ca5c4a83a988adccf7b2897f2"}
Jul 07 17:09:36.321306 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:36.321275    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-gphrw"]
Jul 07 17:09:36.324147 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:36.324130    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-gphrw"
Jul 07 17:09:36.326373 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:36.326347    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 07 17:09:36.326465 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:36.326444    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-rmrkh\""
Jul 07 17:09:36.336759 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:36.336738    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-gphrw"]
Jul 07 17:09:36.430922 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:36.430871    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qnj2\" (UniqueName: \"kubernetes.io/projected/b5856597-5bb9-4e7d-ab73-85d76c5ad596-kube-api-access-9qnj2\") pod \"monitoring-plugin-d5dcff7cf-gphrw\" (UID: \"b5856597-5bb9-4e7d-ab73-85d76c5ad596\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-gphrw"
Jul 07 17:09:36.431045 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:36.430934    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b5856597-5bb9-4e7d-ab73-85d76c5ad596-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-gphrw\" (UID: \"b5856597-5bb9-4e7d-ab73-85d76c5ad596\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-gphrw"
Jul 07 17:09:36.532044 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:36.532001    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qnj2\" (UniqueName: \"kubernetes.io/projected/b5856597-5bb9-4e7d-ab73-85d76c5ad596-kube-api-access-9qnj2\") pod \"monitoring-plugin-d5dcff7cf-gphrw\" (UID: \"b5856597-5bb9-4e7d-ab73-85d76c5ad596\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-gphrw"
Jul 07 17:09:36.532175 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:36.532058    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b5856597-5bb9-4e7d-ab73-85d76c5ad596-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-gphrw\" (UID: \"b5856597-5bb9-4e7d-ab73-85d76c5ad596\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-gphrw"
Jul 07 17:09:36.534815 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:36.534786    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b5856597-5bb9-4e7d-ab73-85d76c5ad596-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-gphrw\" (UID: \"b5856597-5bb9-4e7d-ab73-85d76c5ad596\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-gphrw"
Jul 07 17:09:36.540954 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:36.540931    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qnj2\" (UniqueName: \"kubernetes.io/projected/b5856597-5bb9-4e7d-ab73-85d76c5ad596-kube-api-access-9qnj2\") pod \"monitoring-plugin-d5dcff7cf-gphrw\" (UID: \"b5856597-5bb9-4e7d-ab73-85d76c5ad596\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-gphrw"
Jul 07 17:09:36.635097 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:36.635052    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-gphrw"
Jul 07 17:09:36.780232 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:36.780171    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-gphrw"]
Jul 07 17:09:36.782598 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:09:36.782572    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5856597_5bb9_4e7d_ab73_85d76c5ad596.slice/crio-a3b5f88988ef33ca9c94c693270fdffecd6ba3ce807ba392465096e99f6c1ebc WatchSource:0}: Error finding container a3b5f88988ef33ca9c94c693270fdffecd6ba3ce807ba392465096e99f6c1ebc: Status 404 returned error can't find the container with id a3b5f88988ef33ca9c94c693270fdffecd6ba3ce807ba392465096e99f6c1ebc
Jul 07 17:09:37.551054 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:37.550938    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-gphrw" event={"ID":"b5856597-5bb9-4e7d-ab73-85d76c5ad596","Type":"ContainerStarted","Data":"a3b5f88988ef33ca9c94c693270fdffecd6ba3ce807ba392465096e99f6c1ebc"}
Jul 07 17:09:38.558203 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:38.558117    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ace5af06-9cac-4a3c-826a-c4a33dcbed95","Type":"ContainerStarted","Data":"91c18870f3293b4d92ee1afd07ff140e4f3b37b28e8fc2086e11ffbd27a23a81"}
Jul 07 17:09:38.558203 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:38.558198    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ace5af06-9cac-4a3c-826a-c4a33dcbed95","Type":"ContainerStarted","Data":"86c2bc272b350716d0dac9ebc062e07ea3a24178f1b988a80196191e804e0456"}
Jul 07 17:09:38.558636 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:38.558224    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ace5af06-9cac-4a3c-826a-c4a33dcbed95","Type":"ContainerStarted","Data":"9159ca0524ded607b6e0661d3d8a603b5713a5ddfe84b7a483750666e01cebfd"}
Jul 07 17:09:38.558636 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:38.558247    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ace5af06-9cac-4a3c-826a-c4a33dcbed95","Type":"ContainerStarted","Data":"dea925b60400ef74c5ae2ac44ef5e9c36389eee54bfe7be7f6b182bd301b15ff"}
Jul 07 17:09:38.558636 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:38.558270    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ace5af06-9cac-4a3c-826a-c4a33dcbed95","Type":"ContainerStarted","Data":"a2f59130db609673a191b34a796cbbf6fcd149a8545a58f1cd6c39ad74dbbab3"}
Jul 07 17:09:39.566411 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:39.566338    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ace5af06-9cac-4a3c-826a-c4a33dcbed95","Type":"ContainerStarted","Data":"c83b3b2ffa0b7b65512995b8a5aa57e65cf0a7d3d34b5985521295c48c21b0ff"}
Jul 07 17:09:39.567716 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:39.567685    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-gphrw" event={"ID":"b5856597-5bb9-4e7d-ab73-85d76c5ad596","Type":"ContainerStarted","Data":"292a747b21c5f89a9acb0e27b22eb93d03d4a30906b8baea7185516c422de608"}
Jul 07 17:09:39.567909 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:39.567874    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-gphrw"
Jul 07 17:09:39.572753 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:39.572736    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-gphrw"
Jul 07 17:09:39.604500 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:39.604458    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.8092566469999998 podStartE2EDuration="7.604446944s" totalImagesPullingTime="4.779606933s" totalInitContainerRuntime="15.583364ms" isStatefulPod=false podCreationTimestamp="2026-07-07 17:09:32 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:09:39.603192915 +0000 UTC m=+103.099271422" watchObservedRunningTime="2026-07-07 17:09:39.604446944 +0000 UTC m=+103.100525541"
Jul 07 17:09:39.611819 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:39.611791    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-968bf499f-sdr9g"]
Jul 07 17:09:39.614798 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:39.614779    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-968bf499f-sdr9g"
Jul 07 17:09:39.617411 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:39.617392    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 07 17:09:39.617582 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:39.617564    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-x89tt\""
Jul 07 17:09:39.617659 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:39.617623    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 07 17:09:39.617833 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:39.617805    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 07 17:09:39.617974 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:39.617946    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 07 17:09:39.617974 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:39.617969    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 07 17:09:39.618098 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:39.617971    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 07 17:09:39.618098 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:39.618052    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 07 17:09:39.630855 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:39.630830    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-968bf499f-sdr9g"]
Jul 07 17:09:39.631813 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:39.631775    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-gphrw" podStartSLOduration=1.772565027 podStartE2EDuration="3.631762888s" totalImagesPullingTime="1.859197861s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:09:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:09:39.628057931 +0000 UTC m=+103.124136441" watchObservedRunningTime="2026-07-07 17:09:39.631762888 +0000 UTC m=+103.127841394"
Jul 07 17:09:39.759458 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:39.759434    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9b041bbb-f2d9-4687-a2b8-8315c971d423-service-ca\") pod \"console-968bf499f-sdr9g\" (UID: \"9b041bbb-f2d9-4687-a2b8-8315c971d423\") " pod="openshift-console/console-968bf499f-sdr9g"
Jul 07 17:09:39.759568 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:39.759464    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wc7sx\" (UniqueName: \"kubernetes.io/projected/9b041bbb-f2d9-4687-a2b8-8315c971d423-kube-api-access-wc7sx\") pod \"console-968bf499f-sdr9g\" (UID: \"9b041bbb-f2d9-4687-a2b8-8315c971d423\") " pod="openshift-console/console-968bf499f-sdr9g"
Jul 07 17:09:39.759568 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:39.759489    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9b041bbb-f2d9-4687-a2b8-8315c971d423-console-serving-cert\") pod \"console-968bf499f-sdr9g\" (UID: \"9b041bbb-f2d9-4687-a2b8-8315c971d423\") " pod="openshift-console/console-968bf499f-sdr9g"
Jul 07 17:09:39.759637 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:39.759564    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9b041bbb-f2d9-4687-a2b8-8315c971d423-console-oauth-config\") pod \"console-968bf499f-sdr9g\" (UID: \"9b041bbb-f2d9-4687-a2b8-8315c971d423\") " pod="openshift-console/console-968bf499f-sdr9g"
Jul 07 17:09:39.759637 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:39.759595    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9b041bbb-f2d9-4687-a2b8-8315c971d423-oauth-serving-cert\") pod \"console-968bf499f-sdr9g\" (UID: \"9b041bbb-f2d9-4687-a2b8-8315c971d423\") " pod="openshift-console/console-968bf499f-sdr9g"
Jul 07 17:09:39.759696 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:39.759634    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9b041bbb-f2d9-4687-a2b8-8315c971d423-console-config\") pod \"console-968bf499f-sdr9g\" (UID: \"9b041bbb-f2d9-4687-a2b8-8315c971d423\") " pod="openshift-console/console-968bf499f-sdr9g"
Jul 07 17:09:39.860265 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:39.860216    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9b041bbb-f2d9-4687-a2b8-8315c971d423-console-config\") pod \"console-968bf499f-sdr9g\" (UID: \"9b041bbb-f2d9-4687-a2b8-8315c971d423\") " pod="openshift-console/console-968bf499f-sdr9g"
Jul 07 17:09:39.860265 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:39.860248    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9b041bbb-f2d9-4687-a2b8-8315c971d423-service-ca\") pod \"console-968bf499f-sdr9g\" (UID: \"9b041bbb-f2d9-4687-a2b8-8315c971d423\") " pod="openshift-console/console-968bf499f-sdr9g"
Jul 07 17:09:39.860411 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:39.860266    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wc7sx\" (UniqueName: \"kubernetes.io/projected/9b041bbb-f2d9-4687-a2b8-8315c971d423-kube-api-access-wc7sx\") pod \"console-968bf499f-sdr9g\" (UID: \"9b041bbb-f2d9-4687-a2b8-8315c971d423\") " pod="openshift-console/console-968bf499f-sdr9g"
Jul 07 17:09:39.860491 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:39.860473    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9b041bbb-f2d9-4687-a2b8-8315c971d423-console-serving-cert\") pod \"console-968bf499f-sdr9g\" (UID: \"9b041bbb-f2d9-4687-a2b8-8315c971d423\") " pod="openshift-console/console-968bf499f-sdr9g"
Jul 07 17:09:39.860560 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:39.860522    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9b041bbb-f2d9-4687-a2b8-8315c971d423-console-oauth-config\") pod \"console-968bf499f-sdr9g\" (UID: \"9b041bbb-f2d9-4687-a2b8-8315c971d423\") " pod="openshift-console/console-968bf499f-sdr9g"
Jul 07 17:09:39.860560 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:39.860552    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9b041bbb-f2d9-4687-a2b8-8315c971d423-oauth-serving-cert\") pod \"console-968bf499f-sdr9g\" (UID: \"9b041bbb-f2d9-4687-a2b8-8315c971d423\") " pod="openshift-console/console-968bf499f-sdr9g"
Jul 07 17:09:39.861003 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:39.860975    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9b041bbb-f2d9-4687-a2b8-8315c971d423-service-ca\") pod \"console-968bf499f-sdr9g\" (UID: \"9b041bbb-f2d9-4687-a2b8-8315c971d423\") " pod="openshift-console/console-968bf499f-sdr9g"
Jul 07 17:09:39.861003 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:39.860999    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9b041bbb-f2d9-4687-a2b8-8315c971d423-console-config\") pod \"console-968bf499f-sdr9g\" (UID: \"9b041bbb-f2d9-4687-a2b8-8315c971d423\") " pod="openshift-console/console-968bf499f-sdr9g"
Jul 07 17:09:39.861307 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:39.861184    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9b041bbb-f2d9-4687-a2b8-8315c971d423-oauth-serving-cert\") pod \"console-968bf499f-sdr9g\" (UID: \"9b041bbb-f2d9-4687-a2b8-8315c971d423\") " pod="openshift-console/console-968bf499f-sdr9g"
Jul 07 17:09:39.862989 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:39.862969    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9b041bbb-f2d9-4687-a2b8-8315c971d423-console-oauth-config\") pod \"console-968bf499f-sdr9g\" (UID: \"9b041bbb-f2d9-4687-a2b8-8315c971d423\") " pod="openshift-console/console-968bf499f-sdr9g"
Jul 07 17:09:39.863066 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:39.862987    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9b041bbb-f2d9-4687-a2b8-8315c971d423-console-serving-cert\") pod \"console-968bf499f-sdr9g\" (UID: \"9b041bbb-f2d9-4687-a2b8-8315c971d423\") " pod="openshift-console/console-968bf499f-sdr9g"
Jul 07 17:09:39.870214 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:39.870197    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wc7sx\" (UniqueName: \"kubernetes.io/projected/9b041bbb-f2d9-4687-a2b8-8315c971d423-kube-api-access-wc7sx\") pod \"console-968bf499f-sdr9g\" (UID: \"9b041bbb-f2d9-4687-a2b8-8315c971d423\") " pod="openshift-console/console-968bf499f-sdr9g"
Jul 07 17:09:39.924589 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:39.924563    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-968bf499f-sdr9g"
Jul 07 17:09:40.058243 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:40.058216    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-968bf499f-sdr9g"]
Jul 07 17:09:40.061292 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:09:40.061265    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b041bbb_f2d9_4687_a2b8_8315c971d423.slice/crio-03771aa53ff6c101dc857a7a56395ab4b81a978d6a3686a835c4c1fefeb882af WatchSource:0}: Error finding container 03771aa53ff6c101dc857a7a56395ab4b81a978d6a3686a835c4c1fefeb882af: Status 404 returned error can't find the container with id 03771aa53ff6c101dc857a7a56395ab4b81a978d6a3686a835c4c1fefeb882af
Jul 07 17:09:40.422870 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:40.422845    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-fm4tg"
Jul 07 17:09:40.571681 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:40.571649    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-968bf499f-sdr9g" event={"ID":"9b041bbb-f2d9-4687-a2b8-8315c971d423","Type":"ContainerStarted","Data":"03771aa53ff6c101dc857a7a56395ab4b81a978d6a3686a835c4c1fefeb882af"}
Jul 07 17:09:43.481186 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:43.481160    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-74b85b9dfc-gpbwp"
Jul 07 17:09:44.594851 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:44.594775    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-968bf499f-sdr9g" event={"ID":"9b041bbb-f2d9-4687-a2b8-8315c971d423","Type":"ContainerStarted","Data":"cfff838ae81b7389bfb77a8575e7f7664d9fa2cedb0f4d75b26e712287d39ac7"}
Jul 07 17:09:44.618535 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:44.618489    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-968bf499f-sdr9g" podStartSLOduration=1.456353698 podStartE2EDuration="5.618475357s" totalImagesPullingTime="4.162121659s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:09:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:09:44.618277794 +0000 UTC m=+108.114356300" watchObservedRunningTime="2026-07-07 17:09:44.618475357 +0000 UTC m=+108.114553862"
Jul 07 17:09:45.904462 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:45.904422    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-b4c4c8fd5-x4qmh" podUID="9535133b-ad4d-42dd-a756-c4d4c1f1fdc5" containerName="registry" containerID="cri-o://01caacf2a38ba1127cc178284143d53ec494e4691f27aec37c01925d253ca02b" gracePeriod=30
Jul 07 17:09:46.158671 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:46.158613    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b4c4c8fd5-x4qmh"
Jul 07 17:09:46.311604 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:46.311575    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-ca-trust-extracted\") pod \"9535133b-ad4d-42dd-a756-c4d4c1f1fdc5\" (UID: \"9535133b-ad4d-42dd-a756-c4d4c1f1fdc5\") "
Jul 07 17:09:46.311742 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:46.311611    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-image-registry-private-configuration\") pod \"9535133b-ad4d-42dd-a756-c4d4c1f1fdc5\" (UID: \"9535133b-ad4d-42dd-a756-c4d4c1f1fdc5\") "
Jul 07 17:09:46.311742 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:46.311647    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-installation-pull-secrets\") pod \"9535133b-ad4d-42dd-a756-c4d4c1f1fdc5\" (UID: \"9535133b-ad4d-42dd-a756-c4d4c1f1fdc5\") "
Jul 07 17:09:46.311742 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:46.311694    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9jbvf\" (UniqueName: \"kubernetes.io/projected/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-kube-api-access-9jbvf\") pod \"9535133b-ad4d-42dd-a756-c4d4c1f1fdc5\" (UID: \"9535133b-ad4d-42dd-a756-c4d4c1f1fdc5\") "
Jul 07 17:09:46.311742 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:46.311718    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-bound-sa-token\") pod \"9535133b-ad4d-42dd-a756-c4d4c1f1fdc5\" (UID: \"9535133b-ad4d-42dd-a756-c4d4c1f1fdc5\") "
Jul 07 17:09:46.311973 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:46.311745    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-trusted-ca\") pod \"9535133b-ad4d-42dd-a756-c4d4c1f1fdc5\" (UID: \"9535133b-ad4d-42dd-a756-c4d4c1f1fdc5\") "
Jul 07 17:09:46.311973 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:46.311795    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-registry-certificates\") pod \"9535133b-ad4d-42dd-a756-c4d4c1f1fdc5\" (UID: \"9535133b-ad4d-42dd-a756-c4d4c1f1fdc5\") "
Jul 07 17:09:46.311973 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:46.311823    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-registry-tls\") pod \"9535133b-ad4d-42dd-a756-c4d4c1f1fdc5\" (UID: \"9535133b-ad4d-42dd-a756-c4d4c1f1fdc5\") "
Jul 07 17:09:46.312645 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:46.312565    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "9535133b-ad4d-42dd-a756-c4d4c1f1fdc5" (UID: "9535133b-ad4d-42dd-a756-c4d4c1f1fdc5"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 17:09:46.312873 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:46.312838    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "9535133b-ad4d-42dd-a756-c4d4c1f1fdc5" (UID: "9535133b-ad4d-42dd-a756-c4d4c1f1fdc5"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 17:09:46.314532 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:46.314485    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-kube-api-access-9jbvf" (OuterVolumeSpecName: "kube-api-access-9jbvf") pod "9535133b-ad4d-42dd-a756-c4d4c1f1fdc5" (UID: "9535133b-ad4d-42dd-a756-c4d4c1f1fdc5"). InnerVolumeSpecName "kube-api-access-9jbvf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 17:09:46.314532 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:46.314505    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "9535133b-ad4d-42dd-a756-c4d4c1f1fdc5" (UID: "9535133b-ad4d-42dd-a756-c4d4c1f1fdc5"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 17:09:46.314532 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:46.314527    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "9535133b-ad4d-42dd-a756-c4d4c1f1fdc5" (UID: "9535133b-ad4d-42dd-a756-c4d4c1f1fdc5"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 17:09:46.314682 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:46.314553    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "9535133b-ad4d-42dd-a756-c4d4c1f1fdc5" (UID: "9535133b-ad4d-42dd-a756-c4d4c1f1fdc5"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 17:09:46.314682 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:46.314606    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "9535133b-ad4d-42dd-a756-c4d4c1f1fdc5" (UID: "9535133b-ad4d-42dd-a756-c4d4c1f1fdc5"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 17:09:46.320696 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:46.320678    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "9535133b-ad4d-42dd-a756-c4d4c1f1fdc5" (UID: "9535133b-ad4d-42dd-a756-c4d4c1f1fdc5"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:09:46.412696 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:46.412653    2579 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-ca-trust-extracted\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:09:46.412696 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:46.412674    2579 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-image-registry-private-configuration\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:09:46.412696 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:46.412683    2579 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-installation-pull-secrets\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:09:46.412696 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:46.412692    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9jbvf\" (UniqueName: \"kubernetes.io/projected/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-kube-api-access-9jbvf\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:09:46.412853 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:46.412701    2579 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-bound-sa-token\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:09:46.412853 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:46.412710    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-trusted-ca\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:09:46.412853 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:46.412717    2579 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-registry-certificates\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:09:46.412853 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:46.412727    2579 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5-registry-tls\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:09:46.602413 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:46.602390    2579 generic.go:356] "Generic (PLEG): container finished" podID="9535133b-ad4d-42dd-a756-c4d4c1f1fdc5" containerID="01caacf2a38ba1127cc178284143d53ec494e4691f27aec37c01925d253ca02b" exitCode=0
Jul 07 17:09:46.602494 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:46.602432    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b4c4c8fd5-x4qmh"
Jul 07 17:09:46.602494 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:46.602474    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b4c4c8fd5-x4qmh" event={"ID":"9535133b-ad4d-42dd-a756-c4d4c1f1fdc5","Type":"ContainerDied","Data":"01caacf2a38ba1127cc178284143d53ec494e4691f27aec37c01925d253ca02b"}
Jul 07 17:09:46.602564 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:46.602522    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b4c4c8fd5-x4qmh" event={"ID":"9535133b-ad4d-42dd-a756-c4d4c1f1fdc5","Type":"ContainerDied","Data":"6e360e3765e9d5b6a4c0d865801659f0f60527929eb4c91d17efea57f9f6c203"}
Jul 07 17:09:46.602564 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:46.602552    2579 scope.go:117] "RemoveContainer" containerID="01caacf2a38ba1127cc178284143d53ec494e4691f27aec37c01925d253ca02b"
Jul 07 17:09:46.610673 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:46.610659    2579 scope.go:117] "RemoveContainer" containerID="01caacf2a38ba1127cc178284143d53ec494e4691f27aec37c01925d253ca02b"
Jul 07 17:09:46.610958 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:09:46.610936    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01caacf2a38ba1127cc178284143d53ec494e4691f27aec37c01925d253ca02b\": container with ID starting with 01caacf2a38ba1127cc178284143d53ec494e4691f27aec37c01925d253ca02b not found: ID does not exist" containerID="01caacf2a38ba1127cc178284143d53ec494e4691f27aec37c01925d253ca02b"
Jul 07 17:09:46.611040 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:46.610966    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01caacf2a38ba1127cc178284143d53ec494e4691f27aec37c01925d253ca02b"} err="failed to get container status \"01caacf2a38ba1127cc178284143d53ec494e4691f27aec37c01925d253ca02b\": rpc error: code = NotFound desc = could not find container \"01caacf2a38ba1127cc178284143d53ec494e4691f27aec37c01925d253ca02b\": container with ID starting with 01caacf2a38ba1127cc178284143d53ec494e4691f27aec37c01925d253ca02b not found: ID does not exist"
Jul 07 17:09:46.624111 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:46.624093    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-b4c4c8fd5-x4qmh"]
Jul 07 17:09:46.628399 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:46.628381    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-b4c4c8fd5-x4qmh"]
Jul 07 17:09:47.025730 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:47.025694    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9535133b-ad4d-42dd-a756-c4d4c1f1fdc5" path="/var/lib/kubelet/pods/9535133b-ad4d-42dd-a756-c4d4c1f1fdc5/volumes"
Jul 07 17:09:49.925467 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:49.925433    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-968bf499f-sdr9g"
Jul 07 17:09:49.925867 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:49.925524    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-968bf499f-sdr9g"
Jul 07 17:09:49.930092 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:49.930072    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-968bf499f-sdr9g"
Jul 07 17:09:50.620185 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:50.620161    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-968bf499f-sdr9g"
Jul 07 17:09:55.530578 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:55.530549    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-968bf499f-sdr9g"]
Jul 07 17:09:57.645079 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:57.645043    2579 generic.go:356] "Generic (PLEG): container finished" podID="bb66b2cc-3724-4d3b-a6ff-adc97538bf77" containerID="e0109bd971ff8c0dd5042849e1efd7165af040e78e9f6f5bc07efc8575d92de8" exitCode=0
Jul 07 17:09:57.645448 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:57.645121    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-v4z9l" event={"ID":"bb66b2cc-3724-4d3b-a6ff-adc97538bf77","Type":"ContainerDied","Data":"e0109bd971ff8c0dd5042849e1efd7165af040e78e9f6f5bc07efc8575d92de8"}
Jul 07 17:09:57.645448 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:57.645445    2579 scope.go:117] "RemoveContainer" containerID="e0109bd971ff8c0dd5042849e1efd7165af040e78e9f6f5bc07efc8575d92de8"
Jul 07 17:09:58.650460 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:09:58.650424    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-v4z9l" event={"ID":"bb66b2cc-3724-4d3b-a6ff-adc97538bf77","Type":"ContainerStarted","Data":"27cd8cb1050ef747df407c94f5390d1f59331e34cd6dd5bcaa9071da88c68106"}
Jul 07 17:10:06.766993 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:06.766957    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c398fff8-ec63-4063-b3a4-92bf0aa5df9c-metrics-certs\") pod \"network-metrics-daemon-24rwk\" (UID: \"c398fff8-ec63-4063-b3a4-92bf0aa5df9c\") " pod="openshift-multus/network-metrics-daemon-24rwk"
Jul 07 17:10:06.769319 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:06.769297    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c398fff8-ec63-4063-b3a4-92bf0aa5df9c-metrics-certs\") pod \"network-metrics-daemon-24rwk\" (UID: \"c398fff8-ec63-4063-b3a4-92bf0aa5df9c\") " pod="openshift-multus/network-metrics-daemon-24rwk"
Jul 07 17:10:06.935751 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:06.935720    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-chtgz\""
Jul 07 17:10:06.943177 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:06.943147    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-24rwk"
Jul 07 17:10:07.071734 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:07.071707    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-24rwk"]
Jul 07 17:10:07.073966 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:10:07.073943    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc398fff8_ec63_4063_b3a4_92bf0aa5df9c.slice/crio-195210a74ac9619e2a9b2b1579fb08ef454bb69e0048404be80214a934728919 WatchSource:0}: Error finding container 195210a74ac9619e2a9b2b1579fb08ef454bb69e0048404be80214a934728919: Status 404 returned error can't find the container with id 195210a74ac9619e2a9b2b1579fb08ef454bb69e0048404be80214a934728919
Jul 07 17:10:07.683355 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:07.683307    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-24rwk" event={"ID":"c398fff8-ec63-4063-b3a4-92bf0aa5df9c","Type":"ContainerStarted","Data":"195210a74ac9619e2a9b2b1579fb08ef454bb69e0048404be80214a934728919"}
Jul 07 17:10:08.688809 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:08.688771    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-24rwk" event={"ID":"c398fff8-ec63-4063-b3a4-92bf0aa5df9c","Type":"ContainerStarted","Data":"4aad5277e018c5aa4098673d7947c88aa87cb817338ecd94cd712dc221b93fe5"}
Jul 07 17:10:08.689313 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:08.688824    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-24rwk" event={"ID":"c398fff8-ec63-4063-b3a4-92bf0aa5df9c","Type":"ContainerStarted","Data":"e95119ac30b896324983c26e31775900d7acf9887fc444dc6ff52c890ce92dcb"}
Jul 07 17:10:08.709874 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:08.709829    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-24rwk" podStartSLOduration=129.785241913 podStartE2EDuration="2m10.709814946s" totalImagesPullingTime="924.573033ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:07:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:10:08.708838842 +0000 UTC m=+132.204917350" watchObservedRunningTime="2026-07-07 17:10:08.709814946 +0000 UTC m=+132.205893453"
Jul 07 17:10:20.550745 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:20.550698    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-968bf499f-sdr9g" podUID="9b041bbb-f2d9-4687-a2b8-8315c971d423" containerName="console" containerID="cri-o://cfff838ae81b7389bfb77a8575e7f7664d9fa2cedb0f4d75b26e712287d39ac7" gracePeriod=15
Jul 07 17:10:20.617558 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:20.617530    2579 patch_prober.go:28] interesting pod/console-968bf499f-sdr9g container/console namespace/openshift-console: Readiness probe status=failure output="Get \"https://10.134.0.18:8443/health\": dial tcp 10.134.0.18:8443: connect: connection refused" start-of-body=
Jul 07 17:10:20.617652 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:20.617578    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/console-968bf499f-sdr9g" podUID="9b041bbb-f2d9-4687-a2b8-8315c971d423" containerName="console" probeResult="failure" output="Get \"https://10.134.0.18:8443/health\": dial tcp 10.134.0.18:8443: connect: connection refused"
Jul 07 17:10:20.736141 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:20.736104    2579 generic.go:356] "Generic (PLEG): container finished" podID="9b041bbb-f2d9-4687-a2b8-8315c971d423" containerID="cfff838ae81b7389bfb77a8575e7f7664d9fa2cedb0f4d75b26e712287d39ac7" exitCode=0
Jul 07 17:10:20.736304 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:20.736157    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-968bf499f-sdr9g" event={"ID":"9b041bbb-f2d9-4687-a2b8-8315c971d423","Type":"ContainerDied","Data":"cfff838ae81b7389bfb77a8575e7f7664d9fa2cedb0f4d75b26e712287d39ac7"}
Jul 07 17:10:20.785036 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:20.785009    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-968bf499f-sdr9g"
Jul 07 17:10:20.862710 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:20.862658    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9b041bbb-f2d9-4687-a2b8-8315c971d423-oauth-serving-cert\") pod \"9b041bbb-f2d9-4687-a2b8-8315c971d423\" (UID: \"9b041bbb-f2d9-4687-a2b8-8315c971d423\") "
Jul 07 17:10:20.862710 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:20.862698    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9b041bbb-f2d9-4687-a2b8-8315c971d423-service-ca\") pod \"9b041bbb-f2d9-4687-a2b8-8315c971d423\" (UID: \"9b041bbb-f2d9-4687-a2b8-8315c971d423\") "
Jul 07 17:10:20.862831 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:20.862728    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9b041bbb-f2d9-4687-a2b8-8315c971d423-console-oauth-config\") pod \"9b041bbb-f2d9-4687-a2b8-8315c971d423\" (UID: \"9b041bbb-f2d9-4687-a2b8-8315c971d423\") "
Jul 07 17:10:20.862831 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:20.862749    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wc7sx\" (UniqueName: \"kubernetes.io/projected/9b041bbb-f2d9-4687-a2b8-8315c971d423-kube-api-access-wc7sx\") pod \"9b041bbb-f2d9-4687-a2b8-8315c971d423\" (UID: \"9b041bbb-f2d9-4687-a2b8-8315c971d423\") "
Jul 07 17:10:20.862831 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:20.862769    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9b041bbb-f2d9-4687-a2b8-8315c971d423-console-config\") pod \"9b041bbb-f2d9-4687-a2b8-8315c971d423\" (UID: \"9b041bbb-f2d9-4687-a2b8-8315c971d423\") "
Jul 07 17:10:20.862831 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:20.862794    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9b041bbb-f2d9-4687-a2b8-8315c971d423-console-serving-cert\") pod \"9b041bbb-f2d9-4687-a2b8-8315c971d423\" (UID: \"9b041bbb-f2d9-4687-a2b8-8315c971d423\") "
Jul 07 17:10:20.863167 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:20.863138    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b041bbb-f2d9-4687-a2b8-8315c971d423-service-ca" (OuterVolumeSpecName: "service-ca") pod "9b041bbb-f2d9-4687-a2b8-8315c971d423" (UID: "9b041bbb-f2d9-4687-a2b8-8315c971d423"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 17:10:20.863244 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:20.863160    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b041bbb-f2d9-4687-a2b8-8315c971d423-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "9b041bbb-f2d9-4687-a2b8-8315c971d423" (UID: "9b041bbb-f2d9-4687-a2b8-8315c971d423"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 17:10:20.863244 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:20.863213    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b041bbb-f2d9-4687-a2b8-8315c971d423-console-config" (OuterVolumeSpecName: "console-config") pod "9b041bbb-f2d9-4687-a2b8-8315c971d423" (UID: "9b041bbb-f2d9-4687-a2b8-8315c971d423"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 17:10:20.865075 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:20.865051    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b041bbb-f2d9-4687-a2b8-8315c971d423-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "9b041bbb-f2d9-4687-a2b8-8315c971d423" (UID: "9b041bbb-f2d9-4687-a2b8-8315c971d423"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 17:10:20.865154 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:20.865101    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b041bbb-f2d9-4687-a2b8-8315c971d423-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "9b041bbb-f2d9-4687-a2b8-8315c971d423" (UID: "9b041bbb-f2d9-4687-a2b8-8315c971d423"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 17:10:20.865154 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:20.865137    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b041bbb-f2d9-4687-a2b8-8315c971d423-kube-api-access-wc7sx" (OuterVolumeSpecName: "kube-api-access-wc7sx") pod "9b041bbb-f2d9-4687-a2b8-8315c971d423" (UID: "9b041bbb-f2d9-4687-a2b8-8315c971d423"). InnerVolumeSpecName "kube-api-access-wc7sx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 17:10:20.963639 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:20.963604    2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9b041bbb-f2d9-4687-a2b8-8315c971d423-console-oauth-config\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:10:20.963639 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:20.963639    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wc7sx\" (UniqueName: \"kubernetes.io/projected/9b041bbb-f2d9-4687-a2b8-8315c971d423-kube-api-access-wc7sx\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:10:20.963758 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:20.963649    2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9b041bbb-f2d9-4687-a2b8-8315c971d423-console-config\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:10:20.963758 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:20.963658    2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9b041bbb-f2d9-4687-a2b8-8315c971d423-console-serving-cert\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:10:20.963758 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:20.963666    2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9b041bbb-f2d9-4687-a2b8-8315c971d423-oauth-serving-cert\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:10:20.963758 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:20.963674    2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9b041bbb-f2d9-4687-a2b8-8315c971d423-service-ca\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:10:21.740248 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:21.740212    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-968bf499f-sdr9g"
Jul 07 17:10:21.740248 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:21.740237    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-968bf499f-sdr9g" event={"ID":"9b041bbb-f2d9-4687-a2b8-8315c971d423","Type":"ContainerDied","Data":"03771aa53ff6c101dc857a7a56395ab4b81a978d6a3686a835c4c1fefeb882af"}
Jul 07 17:10:21.740690 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:21.740328    2579 scope.go:117] "RemoveContainer" containerID="cfff838ae81b7389bfb77a8575e7f7664d9fa2cedb0f4d75b26e712287d39ac7"
Jul 07 17:10:21.756936 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:21.756913    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-968bf499f-sdr9g"]
Jul 07 17:10:21.763371 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:21.763351    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-968bf499f-sdr9g"]
Jul 07 17:10:23.026495 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:23.026459    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b041bbb-f2d9-4687-a2b8-8315c971d423" path="/var/lib/kubelet/pods/9b041bbb-f2d9-4687-a2b8-8315c971d423/volumes"
Jul 07 17:10:52.084278 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:52.084199    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 07 17:10:52.084745 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:52.084513    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ace5af06-9cac-4a3c-826a-c4a33dcbed95" containerName="alertmanager" containerID="cri-o://a2f59130db609673a191b34a796cbbf6fcd149a8545a58f1cd6c39ad74dbbab3" gracePeriod=120
Jul 07 17:10:52.084745 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:52.084557    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ace5af06-9cac-4a3c-826a-c4a33dcbed95" containerName="kube-rbac-proxy-metric" containerID="cri-o://91c18870f3293b4d92ee1afd07ff140e4f3b37b28e8fc2086e11ffbd27a23a81" gracePeriod=120
Jul 07 17:10:52.084745 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:52.084597    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ace5af06-9cac-4a3c-826a-c4a33dcbed95" containerName="config-reloader" containerID="cri-o://dea925b60400ef74c5ae2ac44ef5e9c36389eee54bfe7be7f6b182bd301b15ff" gracePeriod=120
Jul 07 17:10:52.084745 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:52.084600    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ace5af06-9cac-4a3c-826a-c4a33dcbed95" containerName="kube-rbac-proxy-web" containerID="cri-o://9159ca0524ded607b6e0661d3d8a603b5713a5ddfe84b7a483750666e01cebfd" gracePeriod=120
Jul 07 17:10:52.084745 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:52.084625    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ace5af06-9cac-4a3c-826a-c4a33dcbed95" containerName="kube-rbac-proxy" containerID="cri-o://86c2bc272b350716d0dac9ebc062e07ea3a24178f1b988a80196191e804e0456" gracePeriod=120
Jul 07 17:10:52.084745 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:52.084695    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ace5af06-9cac-4a3c-826a-c4a33dcbed95" containerName="prom-label-proxy" containerID="cri-o://c83b3b2ffa0b7b65512995b8a5aa57e65cf0a7d3d34b5985521295c48c21b0ff" gracePeriod=120
Jul 07 17:10:52.854170 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:52.854136    2579 generic.go:356] "Generic (PLEG): container finished" podID="ace5af06-9cac-4a3c-826a-c4a33dcbed95" containerID="c83b3b2ffa0b7b65512995b8a5aa57e65cf0a7d3d34b5985521295c48c21b0ff" exitCode=0
Jul 07 17:10:52.854170 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:52.854161    2579 generic.go:356] "Generic (PLEG): container finished" podID="ace5af06-9cac-4a3c-826a-c4a33dcbed95" containerID="86c2bc272b350716d0dac9ebc062e07ea3a24178f1b988a80196191e804e0456" exitCode=0
Jul 07 17:10:52.854170 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:52.854168    2579 generic.go:356] "Generic (PLEG): container finished" podID="ace5af06-9cac-4a3c-826a-c4a33dcbed95" containerID="dea925b60400ef74c5ae2ac44ef5e9c36389eee54bfe7be7f6b182bd301b15ff" exitCode=0
Jul 07 17:10:52.854170 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:52.854173    2579 generic.go:356] "Generic (PLEG): container finished" podID="ace5af06-9cac-4a3c-826a-c4a33dcbed95" containerID="a2f59130db609673a191b34a796cbbf6fcd149a8545a58f1cd6c39ad74dbbab3" exitCode=0
Jul 07 17:10:52.854422 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:52.854204    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ace5af06-9cac-4a3c-826a-c4a33dcbed95","Type":"ContainerDied","Data":"c83b3b2ffa0b7b65512995b8a5aa57e65cf0a7d3d34b5985521295c48c21b0ff"}
Jul 07 17:10:52.854422 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:52.854286    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ace5af06-9cac-4a3c-826a-c4a33dcbed95","Type":"ContainerDied","Data":"86c2bc272b350716d0dac9ebc062e07ea3a24178f1b988a80196191e804e0456"}
Jul 07 17:10:52.854422 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:52.854313    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ace5af06-9cac-4a3c-826a-c4a33dcbed95","Type":"ContainerDied","Data":"dea925b60400ef74c5ae2ac44ef5e9c36389eee54bfe7be7f6b182bd301b15ff"}
Jul 07 17:10:52.854422 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:52.854338    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ace5af06-9cac-4a3c-826a-c4a33dcbed95","Type":"ContainerDied","Data":"a2f59130db609673a191b34a796cbbf6fcd149a8545a58f1cd6c39ad74dbbab3"}
Jul 07 17:10:53.317983 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.317961    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:10:53.338143 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.338119    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-dd4f44b6d-khn84"]
Jul 07 17:10:53.338357 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.338346    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b041bbb-f2d9-4687-a2b8-8315c971d423" containerName="console"
Jul 07 17:10:53.338402 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.338359    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b041bbb-f2d9-4687-a2b8-8315c971d423" containerName="console"
Jul 07 17:10:53.338402 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.338368    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ace5af06-9cac-4a3c-826a-c4a33dcbed95" containerName="kube-rbac-proxy-web"
Jul 07 17:10:53.338402 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.338373    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ace5af06-9cac-4a3c-826a-c4a33dcbed95" containerName="kube-rbac-proxy-web"
Jul 07 17:10:53.338402 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.338382    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ace5af06-9cac-4a3c-826a-c4a33dcbed95" containerName="alertmanager"
Jul 07 17:10:53.338402 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.338388    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ace5af06-9cac-4a3c-826a-c4a33dcbed95" containerName="alertmanager"
Jul 07 17:10:53.338402 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.338395    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ace5af06-9cac-4a3c-826a-c4a33dcbed95" containerName="kube-rbac-proxy"
Jul 07 17:10:53.338402 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.338400    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ace5af06-9cac-4a3c-826a-c4a33dcbed95" containerName="kube-rbac-proxy"
Jul 07 17:10:53.338594 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.338409    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ace5af06-9cac-4a3c-826a-c4a33dcbed95" containerName="config-reloader"
Jul 07 17:10:53.338594 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.338414    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ace5af06-9cac-4a3c-826a-c4a33dcbed95" containerName="config-reloader"
Jul 07 17:10:53.338594 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.338423    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ace5af06-9cac-4a3c-826a-c4a33dcbed95" containerName="kube-rbac-proxy-metric"
Jul 07 17:10:53.338594 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.338429    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ace5af06-9cac-4a3c-826a-c4a33dcbed95" containerName="kube-rbac-proxy-metric"
Jul 07 17:10:53.338594 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.338435    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9535133b-ad4d-42dd-a756-c4d4c1f1fdc5" containerName="registry"
Jul 07 17:10:53.338594 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.338440    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9535133b-ad4d-42dd-a756-c4d4c1f1fdc5" containerName="registry"
Jul 07 17:10:53.338594 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.338446    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ace5af06-9cac-4a3c-826a-c4a33dcbed95" containerName="init-config-reloader"
Jul 07 17:10:53.338594 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.338451    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ace5af06-9cac-4a3c-826a-c4a33dcbed95" containerName="init-config-reloader"
Jul 07 17:10:53.338594 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.338460    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ace5af06-9cac-4a3c-826a-c4a33dcbed95" containerName="prom-label-proxy"
Jul 07 17:10:53.338594 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.338465    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ace5af06-9cac-4a3c-826a-c4a33dcbed95" containerName="prom-label-proxy"
Jul 07 17:10:53.338594 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.338500    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ace5af06-9cac-4a3c-826a-c4a33dcbed95" containerName="kube-rbac-proxy-web"
Jul 07 17:10:53.338594 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.338506    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ace5af06-9cac-4a3c-826a-c4a33dcbed95" containerName="kube-rbac-proxy-metric"
Jul 07 17:10:53.338594 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.338512    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ace5af06-9cac-4a3c-826a-c4a33dcbed95" containerName="kube-rbac-proxy"
Jul 07 17:10:53.338594 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.338518    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ace5af06-9cac-4a3c-826a-c4a33dcbed95" containerName="config-reloader"
Jul 07 17:10:53.338594 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.338523    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ace5af06-9cac-4a3c-826a-c4a33dcbed95" containerName="prom-label-proxy"
Jul 07 17:10:53.338594 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.338528    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b041bbb-f2d9-4687-a2b8-8315c971d423" containerName="console"
Jul 07 17:10:53.338594 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.338532    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ace5af06-9cac-4a3c-826a-c4a33dcbed95" containerName="alertmanager"
Jul 07 17:10:53.338594 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.338537    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="9535133b-ad4d-42dd-a756-c4d4c1f1fdc5" containerName="registry"
Jul 07 17:10:53.340112 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.340097    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dd4f44b6d-khn84"
Jul 07 17:10:53.345391 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.345370    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-x89tt\""
Jul 07 17:10:53.345491 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.345451    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 07 17:10:53.345649 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.345629    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 07 17:10:53.345794 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.345779    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 07 17:10:53.345920 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.345883    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 07 17:10:53.346021 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.346005    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 07 17:10:53.346090 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.346040    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 07 17:10:53.346211 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.346145    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 07 17:10:53.349672 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.349656    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 07 17:10:53.356663 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.356640    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-dd4f44b6d-khn84"]
Jul 07 17:10:53.397657 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.397625    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ace5af06-9cac-4a3c-826a-c4a33dcbed95-web-config\") pod \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\" (UID: \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\") "
Jul 07 17:10:53.397826 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.397685    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ace5af06-9cac-4a3c-826a-c4a33dcbed95-metrics-client-ca\") pod \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\" (UID: \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\") "
Jul 07 17:10:53.397826 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.397712    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vb5bm\" (UniqueName: \"kubernetes.io/projected/ace5af06-9cac-4a3c-826a-c4a33dcbed95-kube-api-access-vb5bm\") pod \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\" (UID: \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\") "
Jul 07 17:10:53.397826 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.397738    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ace5af06-9cac-4a3c-826a-c4a33dcbed95-alertmanager-trusted-ca-bundle\") pod \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\" (UID: \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\") "
Jul 07 17:10:53.397826 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.397765    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ace5af06-9cac-4a3c-826a-c4a33dcbed95-secret-alertmanager-kube-rbac-proxy-web\") pod \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\" (UID: \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\") "
Jul 07 17:10:53.397826 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.397791    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ace5af06-9cac-4a3c-826a-c4a33dcbed95-config-out\") pod \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\" (UID: \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\") "
Jul 07 17:10:53.397826 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.397822    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ace5af06-9cac-4a3c-826a-c4a33dcbed95-secret-alertmanager-kube-rbac-proxy\") pod \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\" (UID: \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\") "
Jul 07 17:10:53.398174 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.397866    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ace5af06-9cac-4a3c-826a-c4a33dcbed95-tls-assets\") pod \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\" (UID: \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\") "
Jul 07 17:10:53.398174 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.397918    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ace5af06-9cac-4a3c-826a-c4a33dcbed95-alertmanager-main-db\") pod \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\" (UID: \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\") "
Jul 07 17:10:53.398174 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.397942    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ace5af06-9cac-4a3c-826a-c4a33dcbed95-config-volume\") pod \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\" (UID: \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\") "
Jul 07 17:10:53.398174 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.397971    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ace5af06-9cac-4a3c-826a-c4a33dcbed95-secret-alertmanager-kube-rbac-proxy-metric\") pod \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\" (UID: \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\") "
Jul 07 17:10:53.398174 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.398004    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ace5af06-9cac-4a3c-826a-c4a33dcbed95-cluster-tls-config\") pod \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\" (UID: \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\") "
Jul 07 17:10:53.398174 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.398037    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ace5af06-9cac-4a3c-826a-c4a33dcbed95-secret-alertmanager-main-tls\") pod \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\" (UID: \"ace5af06-9cac-4a3c-826a-c4a33dcbed95\") "
Jul 07 17:10:53.398174 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.398134    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4p9l8\" (UniqueName: \"kubernetes.io/projected/3d695934-377d-4c84-8e9c-e2c9bad63689-kube-api-access-4p9l8\") pod \"console-dd4f44b6d-khn84\" (UID: \"3d695934-377d-4c84-8e9c-e2c9bad63689\") " pod="openshift-console/console-dd4f44b6d-khn84"
Jul 07 17:10:53.398174 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.398171    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3d695934-377d-4c84-8e9c-e2c9bad63689-console-config\") pod \"console-dd4f44b6d-khn84\" (UID: \"3d695934-377d-4c84-8e9c-e2c9bad63689\") " pod="openshift-console/console-dd4f44b6d-khn84"
Jul 07 17:10:53.398736 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.398172    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ace5af06-9cac-4a3c-826a-c4a33dcbed95-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "ace5af06-9cac-4a3c-826a-c4a33dcbed95" (UID: "ace5af06-9cac-4a3c-826a-c4a33dcbed95"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 17:10:53.398736 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.398239    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3d695934-377d-4c84-8e9c-e2c9bad63689-console-serving-cert\") pod \"console-dd4f44b6d-khn84\" (UID: \"3d695934-377d-4c84-8e9c-e2c9bad63689\") " pod="openshift-console/console-dd4f44b6d-khn84"
Jul 07 17:10:53.398736 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.398266    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3d695934-377d-4c84-8e9c-e2c9bad63689-service-ca\") pod \"console-dd4f44b6d-khn84\" (UID: \"3d695934-377d-4c84-8e9c-e2c9bad63689\") " pod="openshift-console/console-dd4f44b6d-khn84"
Jul 07 17:10:53.398736 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.398339    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3d695934-377d-4c84-8e9c-e2c9bad63689-console-oauth-config\") pod \"console-dd4f44b6d-khn84\" (UID: \"3d695934-377d-4c84-8e9c-e2c9bad63689\") " pod="openshift-console/console-dd4f44b6d-khn84"
Jul 07 17:10:53.398736 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.398362    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3d695934-377d-4c84-8e9c-e2c9bad63689-oauth-serving-cert\") pod \"console-dd4f44b6d-khn84\" (UID: \"3d695934-377d-4c84-8e9c-e2c9bad63689\") " pod="openshift-console/console-dd4f44b6d-khn84"
Jul 07 17:10:53.398736 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.398406    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3d695934-377d-4c84-8e9c-e2c9bad63689-trusted-ca-bundle\") pod \"console-dd4f44b6d-khn84\" (UID: \"3d695934-377d-4c84-8e9c-e2c9bad63689\") " pod="openshift-console/console-dd4f44b6d-khn84"
Jul 07 17:10:53.398736 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.398462    2579 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ace5af06-9cac-4a3c-826a-c4a33dcbed95-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:10:53.398736 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.398488    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ace5af06-9cac-4a3c-826a-c4a33dcbed95-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "ace5af06-9cac-4a3c-826a-c4a33dcbed95" (UID: "ace5af06-9cac-4a3c-826a-c4a33dcbed95"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:10:53.398736 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.398707    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ace5af06-9cac-4a3c-826a-c4a33dcbed95-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "ace5af06-9cac-4a3c-826a-c4a33dcbed95" (UID: "ace5af06-9cac-4a3c-826a-c4a33dcbed95"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 17:10:53.402464 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.402430    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ace5af06-9cac-4a3c-826a-c4a33dcbed95-kube-api-access-vb5bm" (OuterVolumeSpecName: "kube-api-access-vb5bm") pod "ace5af06-9cac-4a3c-826a-c4a33dcbed95" (UID: "ace5af06-9cac-4a3c-826a-c4a33dcbed95"). InnerVolumeSpecName "kube-api-access-vb5bm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 17:10:53.402585 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.402460    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ace5af06-9cac-4a3c-826a-c4a33dcbed95-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "ace5af06-9cac-4a3c-826a-c4a33dcbed95" (UID: "ace5af06-9cac-4a3c-826a-c4a33dcbed95"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 17:10:53.402647 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.402584    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ace5af06-9cac-4a3c-826a-c4a33dcbed95-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "ace5af06-9cac-4a3c-826a-c4a33dcbed95" (UID: "ace5af06-9cac-4a3c-826a-c4a33dcbed95"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 17:10:53.402647 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.402624    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ace5af06-9cac-4a3c-826a-c4a33dcbed95-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "ace5af06-9cac-4a3c-826a-c4a33dcbed95" (UID: "ace5af06-9cac-4a3c-826a-c4a33dcbed95"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 17:10:53.402777 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.402760    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ace5af06-9cac-4a3c-826a-c4a33dcbed95-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "ace5af06-9cac-4a3c-826a-c4a33dcbed95" (UID: "ace5af06-9cac-4a3c-826a-c4a33dcbed95"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 17:10:53.403286 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.403252    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ace5af06-9cac-4a3c-826a-c4a33dcbed95-config-out" (OuterVolumeSpecName: "config-out") pod "ace5af06-9cac-4a3c-826a-c4a33dcbed95" (UID: "ace5af06-9cac-4a3c-826a-c4a33dcbed95"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:10:53.403492 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.403467    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ace5af06-9cac-4a3c-826a-c4a33dcbed95-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "ace5af06-9cac-4a3c-826a-c4a33dcbed95" (UID: "ace5af06-9cac-4a3c-826a-c4a33dcbed95"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 17:10:53.403608 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.403497    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ace5af06-9cac-4a3c-826a-c4a33dcbed95-config-volume" (OuterVolumeSpecName: "config-volume") pod "ace5af06-9cac-4a3c-826a-c4a33dcbed95" (UID: "ace5af06-9cac-4a3c-826a-c4a33dcbed95"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 17:10:53.405963 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.405860    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ace5af06-9cac-4a3c-826a-c4a33dcbed95-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "ace5af06-9cac-4a3c-826a-c4a33dcbed95" (UID: "ace5af06-9cac-4a3c-826a-c4a33dcbed95"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 17:10:53.411551 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.411531    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ace5af06-9cac-4a3c-826a-c4a33dcbed95-web-config" (OuterVolumeSpecName: "web-config") pod "ace5af06-9cac-4a3c-826a-c4a33dcbed95" (UID: "ace5af06-9cac-4a3c-826a-c4a33dcbed95"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 17:10:53.498926 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.498874    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3d695934-377d-4c84-8e9c-e2c9bad63689-trusted-ca-bundle\") pod \"console-dd4f44b6d-khn84\" (UID: \"3d695934-377d-4c84-8e9c-e2c9bad63689\") " pod="openshift-console/console-dd4f44b6d-khn84"
Jul 07 17:10:53.499119 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.498934    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4p9l8\" (UniqueName: \"kubernetes.io/projected/3d695934-377d-4c84-8e9c-e2c9bad63689-kube-api-access-4p9l8\") pod \"console-dd4f44b6d-khn84\" (UID: \"3d695934-377d-4c84-8e9c-e2c9bad63689\") " pod="openshift-console/console-dd4f44b6d-khn84"
Jul 07 17:10:53.499119 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.498955    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3d695934-377d-4c84-8e9c-e2c9bad63689-console-config\") pod \"console-dd4f44b6d-khn84\" (UID: \"3d695934-377d-4c84-8e9c-e2c9bad63689\") " pod="openshift-console/console-dd4f44b6d-khn84"
Jul 07 17:10:53.499119 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.498982    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3d695934-377d-4c84-8e9c-e2c9bad63689-console-serving-cert\") pod \"console-dd4f44b6d-khn84\" (UID: \"3d695934-377d-4c84-8e9c-e2c9bad63689\") " pod="openshift-console/console-dd4f44b6d-khn84"
Jul 07 17:10:53.499119 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.498996    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3d695934-377d-4c84-8e9c-e2c9bad63689-service-ca\") pod \"console-dd4f44b6d-khn84\" (UID: \"3d695934-377d-4c84-8e9c-e2c9bad63689\") " pod="openshift-console/console-dd4f44b6d-khn84"
Jul 07 17:10:53.499306 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.499150    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3d695934-377d-4c84-8e9c-e2c9bad63689-console-oauth-config\") pod \"console-dd4f44b6d-khn84\" (UID: \"3d695934-377d-4c84-8e9c-e2c9bad63689\") " pod="openshift-console/console-dd4f44b6d-khn84"
Jul 07 17:10:53.499306 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.499192    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3d695934-377d-4c84-8e9c-e2c9bad63689-oauth-serving-cert\") pod \"console-dd4f44b6d-khn84\" (UID: \"3d695934-377d-4c84-8e9c-e2c9bad63689\") " pod="openshift-console/console-dd4f44b6d-khn84"
Jul 07 17:10:53.499306 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.499266    2579 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ace5af06-9cac-4a3c-826a-c4a33dcbed95-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:10:53.499306 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.499283    2579 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ace5af06-9cac-4a3c-826a-c4a33dcbed95-config-out\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:10:53.499306 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.499299    2579 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ace5af06-9cac-4a3c-826a-c4a33dcbed95-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:10:53.499534 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.499315    2579 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ace5af06-9cac-4a3c-826a-c4a33dcbed95-tls-assets\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:10:53.499534 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.499331    2579 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ace5af06-9cac-4a3c-826a-c4a33dcbed95-alertmanager-main-db\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:10:53.499534 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.499345    2579 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ace5af06-9cac-4a3c-826a-c4a33dcbed95-config-volume\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:10:53.499534 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.499359    2579 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ace5af06-9cac-4a3c-826a-c4a33dcbed95-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:10:53.499534 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.499373    2579 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ace5af06-9cac-4a3c-826a-c4a33dcbed95-cluster-tls-config\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:10:53.499534 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.499386    2579 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ace5af06-9cac-4a3c-826a-c4a33dcbed95-secret-alertmanager-main-tls\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:10:53.499534 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.499398    2579 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ace5af06-9cac-4a3c-826a-c4a33dcbed95-web-config\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:10:53.499534 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.499412    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ace5af06-9cac-4a3c-826a-c4a33dcbed95-metrics-client-ca\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:10:53.499534 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.499430    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vb5bm\" (UniqueName: \"kubernetes.io/projected/ace5af06-9cac-4a3c-826a-c4a33dcbed95-kube-api-access-vb5bm\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:10:53.499851 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.499735    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3d695934-377d-4c84-8e9c-e2c9bad63689-service-ca\") pod \"console-dd4f44b6d-khn84\" (UID: \"3d695934-377d-4c84-8e9c-e2c9bad63689\") " pod="openshift-console/console-dd4f44b6d-khn84"
Jul 07 17:10:53.499851 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.499735    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3d695934-377d-4c84-8e9c-e2c9bad63689-console-config\") pod \"console-dd4f44b6d-khn84\" (UID: \"3d695934-377d-4c84-8e9c-e2c9bad63689\") " pod="openshift-console/console-dd4f44b6d-khn84"
Jul 07 17:10:53.500027 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.500004    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3d695934-377d-4c84-8e9c-e2c9bad63689-trusted-ca-bundle\") pod \"console-dd4f44b6d-khn84\" (UID: \"3d695934-377d-4c84-8e9c-e2c9bad63689\") " pod="openshift-console/console-dd4f44b6d-khn84"
Jul 07 17:10:53.500240 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.500218    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3d695934-377d-4c84-8e9c-e2c9bad63689-oauth-serving-cert\") pod \"console-dd4f44b6d-khn84\" (UID: \"3d695934-377d-4c84-8e9c-e2c9bad63689\") " pod="openshift-console/console-dd4f44b6d-khn84"
Jul 07 17:10:53.501554 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.501532    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3d695934-377d-4c84-8e9c-e2c9bad63689-console-serving-cert\") pod \"console-dd4f44b6d-khn84\" (UID: \"3d695934-377d-4c84-8e9c-e2c9bad63689\") " pod="openshift-console/console-dd4f44b6d-khn84"
Jul 07 17:10:53.501636 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.501565    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3d695934-377d-4c84-8e9c-e2c9bad63689-console-oauth-config\") pod \"console-dd4f44b6d-khn84\" (UID: \"3d695934-377d-4c84-8e9c-e2c9bad63689\") " pod="openshift-console/console-dd4f44b6d-khn84"
Jul 07 17:10:53.507366 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.507349    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4p9l8\" (UniqueName: \"kubernetes.io/projected/3d695934-377d-4c84-8e9c-e2c9bad63689-kube-api-access-4p9l8\") pod \"console-dd4f44b6d-khn84\" (UID: \"3d695934-377d-4c84-8e9c-e2c9bad63689\") " pod="openshift-console/console-dd4f44b6d-khn84"
Jul 07 17:10:53.649236 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.649146    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dd4f44b6d-khn84"
Jul 07 17:10:53.777628 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.777606    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-dd4f44b6d-khn84"]
Jul 07 17:10:53.780181 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:10:53.780154    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d695934_377d_4c84_8e9c_e2c9bad63689.slice/crio-3e34f674b15860e2d8df2be4e65cc5c17a65bb36a9467c9a077d13072f033410 WatchSource:0}: Error finding container 3e34f674b15860e2d8df2be4e65cc5c17a65bb36a9467c9a077d13072f033410: Status 404 returned error can't find the container with id 3e34f674b15860e2d8df2be4e65cc5c17a65bb36a9467c9a077d13072f033410
Jul 07 17:10:53.864287 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.863989    2579 generic.go:356] "Generic (PLEG): container finished" podID="ace5af06-9cac-4a3c-826a-c4a33dcbed95" containerID="91c18870f3293b4d92ee1afd07ff140e4f3b37b28e8fc2086e11ffbd27a23a81" exitCode=0
Jul 07 17:10:53.864287 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.864018    2579 generic.go:356] "Generic (PLEG): container finished" podID="ace5af06-9cac-4a3c-826a-c4a33dcbed95" containerID="9159ca0524ded607b6e0661d3d8a603b5713a5ddfe84b7a483750666e01cebfd" exitCode=0
Jul 07 17:10:53.864287 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.864098    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:10:53.864287 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.864115    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ace5af06-9cac-4a3c-826a-c4a33dcbed95","Type":"ContainerDied","Data":"91c18870f3293b4d92ee1afd07ff140e4f3b37b28e8fc2086e11ffbd27a23a81"}
Jul 07 17:10:53.864287 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.864188    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ace5af06-9cac-4a3c-826a-c4a33dcbed95","Type":"ContainerDied","Data":"9159ca0524ded607b6e0661d3d8a603b5713a5ddfe84b7a483750666e01cebfd"}
Jul 07 17:10:53.864287 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.864229    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ace5af06-9cac-4a3c-826a-c4a33dcbed95","Type":"ContainerDied","Data":"0a1113aacfbf15aff401c3588181c6857867416f061ef8b799085d0382eb1a82"}
Jul 07 17:10:53.864287 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.864274    2579 scope.go:117] "RemoveContainer" containerID="c83b3b2ffa0b7b65512995b8a5aa57e65cf0a7d3d34b5985521295c48c21b0ff"
Jul 07 17:10:53.866252 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.866215    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dd4f44b6d-khn84" event={"ID":"3d695934-377d-4c84-8e9c-e2c9bad63689","Type":"ContainerStarted","Data":"8ccfa8fa7c4e87a0aeb2d22d92803918c821934da7c10b0597bd93c96acfdfca"}
Jul 07 17:10:53.866375 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.866280    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dd4f44b6d-khn84" event={"ID":"3d695934-377d-4c84-8e9c-e2c9bad63689","Type":"ContainerStarted","Data":"3e34f674b15860e2d8df2be4e65cc5c17a65bb36a9467c9a077d13072f033410"}
Jul 07 17:10:53.872813 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.872788    2579 scope.go:117] "RemoveContainer" containerID="91c18870f3293b4d92ee1afd07ff140e4f3b37b28e8fc2086e11ffbd27a23a81"
Jul 07 17:10:53.880311 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.880292    2579 scope.go:117] "RemoveContainer" containerID="86c2bc272b350716d0dac9ebc062e07ea3a24178f1b988a80196191e804e0456"
Jul 07 17:10:53.888469 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.888443    2579 scope.go:117] "RemoveContainer" containerID="9159ca0524ded607b6e0661d3d8a603b5713a5ddfe84b7a483750666e01cebfd"
Jul 07 17:10:53.890180 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.890138    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-dd4f44b6d-khn84" podStartSLOduration=0.890123674 podStartE2EDuration="890.123674ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:10:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:10:53.888714221 +0000 UTC m=+177.384792746" watchObservedRunningTime="2026-07-07 17:10:53.890123674 +0000 UTC m=+177.386202182"
Jul 07 17:10:53.896734 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.896711    2579 scope.go:117] "RemoveContainer" containerID="dea925b60400ef74c5ae2ac44ef5e9c36389eee54bfe7be7f6b182bd301b15ff"
Jul 07 17:10:53.904071 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.904051    2579 scope.go:117] "RemoveContainer" containerID="a2f59130db609673a191b34a796cbbf6fcd149a8545a58f1cd6c39ad74dbbab3"
Jul 07 17:10:53.906493 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.906472    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 07 17:10:53.911328 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.911307    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 07 17:10:53.913019 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.913002    2579 scope.go:117] "RemoveContainer" containerID="e3f085ed8401f7f5efbed6861d5b8fd8149cf59ca5c4a83a988adccf7b2897f2"
Jul 07 17:10:53.919573 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.919547    2579 scope.go:117] "RemoveContainer" containerID="c83b3b2ffa0b7b65512995b8a5aa57e65cf0a7d3d34b5985521295c48c21b0ff"
Jul 07 17:10:53.919837 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:10:53.919817    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c83b3b2ffa0b7b65512995b8a5aa57e65cf0a7d3d34b5985521295c48c21b0ff\": container with ID starting with c83b3b2ffa0b7b65512995b8a5aa57e65cf0a7d3d34b5985521295c48c21b0ff not found: ID does not exist" containerID="c83b3b2ffa0b7b65512995b8a5aa57e65cf0a7d3d34b5985521295c48c21b0ff"
Jul 07 17:10:53.919883 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.919848    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c83b3b2ffa0b7b65512995b8a5aa57e65cf0a7d3d34b5985521295c48c21b0ff"} err="failed to get container status \"c83b3b2ffa0b7b65512995b8a5aa57e65cf0a7d3d34b5985521295c48c21b0ff\": rpc error: code = NotFound desc = could not find container \"c83b3b2ffa0b7b65512995b8a5aa57e65cf0a7d3d34b5985521295c48c21b0ff\": container with ID starting with c83b3b2ffa0b7b65512995b8a5aa57e65cf0a7d3d34b5985521295c48c21b0ff not found: ID does not exist"
Jul 07 17:10:53.919883 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.919867    2579 scope.go:117] "RemoveContainer" containerID="91c18870f3293b4d92ee1afd07ff140e4f3b37b28e8fc2086e11ffbd27a23a81"
Jul 07 17:10:53.920135 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:10:53.920116    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91c18870f3293b4d92ee1afd07ff140e4f3b37b28e8fc2086e11ffbd27a23a81\": container with ID starting with 91c18870f3293b4d92ee1afd07ff140e4f3b37b28e8fc2086e11ffbd27a23a81 not found: ID does not exist" containerID="91c18870f3293b4d92ee1afd07ff140e4f3b37b28e8fc2086e11ffbd27a23a81"
Jul 07 17:10:53.920201 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.920145    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91c18870f3293b4d92ee1afd07ff140e4f3b37b28e8fc2086e11ffbd27a23a81"} err="failed to get container status \"91c18870f3293b4d92ee1afd07ff140e4f3b37b28e8fc2086e11ffbd27a23a81\": rpc error: code = NotFound desc = could not find container \"91c18870f3293b4d92ee1afd07ff140e4f3b37b28e8fc2086e11ffbd27a23a81\": container with ID starting with 91c18870f3293b4d92ee1afd07ff140e4f3b37b28e8fc2086e11ffbd27a23a81 not found: ID does not exist"
Jul 07 17:10:53.920201 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.920168    2579 scope.go:117] "RemoveContainer" containerID="86c2bc272b350716d0dac9ebc062e07ea3a24178f1b988a80196191e804e0456"
Jul 07 17:10:53.920400 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:10:53.920383    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86c2bc272b350716d0dac9ebc062e07ea3a24178f1b988a80196191e804e0456\": container with ID starting with 86c2bc272b350716d0dac9ebc062e07ea3a24178f1b988a80196191e804e0456 not found: ID does not exist" containerID="86c2bc272b350716d0dac9ebc062e07ea3a24178f1b988a80196191e804e0456"
Jul 07 17:10:53.920439 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.920404    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86c2bc272b350716d0dac9ebc062e07ea3a24178f1b988a80196191e804e0456"} err="failed to get container status \"86c2bc272b350716d0dac9ebc062e07ea3a24178f1b988a80196191e804e0456\": rpc error: code = NotFound desc = could not find container \"86c2bc272b350716d0dac9ebc062e07ea3a24178f1b988a80196191e804e0456\": container with ID starting with 86c2bc272b350716d0dac9ebc062e07ea3a24178f1b988a80196191e804e0456 not found: ID does not exist"
Jul 07 17:10:53.920439 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.920417    2579 scope.go:117] "RemoveContainer" containerID="9159ca0524ded607b6e0661d3d8a603b5713a5ddfe84b7a483750666e01cebfd"
Jul 07 17:10:53.920666 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:10:53.920651    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9159ca0524ded607b6e0661d3d8a603b5713a5ddfe84b7a483750666e01cebfd\": container with ID starting with 9159ca0524ded607b6e0661d3d8a603b5713a5ddfe84b7a483750666e01cebfd not found: ID does not exist" containerID="9159ca0524ded607b6e0661d3d8a603b5713a5ddfe84b7a483750666e01cebfd"
Jul 07 17:10:53.920707 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.920669    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9159ca0524ded607b6e0661d3d8a603b5713a5ddfe84b7a483750666e01cebfd"} err="failed to get container status \"9159ca0524ded607b6e0661d3d8a603b5713a5ddfe84b7a483750666e01cebfd\": rpc error: code = NotFound desc = could not find container \"9159ca0524ded607b6e0661d3d8a603b5713a5ddfe84b7a483750666e01cebfd\": container with ID starting with 9159ca0524ded607b6e0661d3d8a603b5713a5ddfe84b7a483750666e01cebfd not found: ID does not exist"
Jul 07 17:10:53.920707 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.920682    2579 scope.go:117] "RemoveContainer" containerID="dea925b60400ef74c5ae2ac44ef5e9c36389eee54bfe7be7f6b182bd301b15ff"
Jul 07 17:10:53.920873 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:10:53.920856    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dea925b60400ef74c5ae2ac44ef5e9c36389eee54bfe7be7f6b182bd301b15ff\": container with ID starting with dea925b60400ef74c5ae2ac44ef5e9c36389eee54bfe7be7f6b182bd301b15ff not found: ID does not exist" containerID="dea925b60400ef74c5ae2ac44ef5e9c36389eee54bfe7be7f6b182bd301b15ff"
Jul 07 17:10:53.920965 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.920875    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dea925b60400ef74c5ae2ac44ef5e9c36389eee54bfe7be7f6b182bd301b15ff"} err="failed to get container status \"dea925b60400ef74c5ae2ac44ef5e9c36389eee54bfe7be7f6b182bd301b15ff\": rpc error: code = NotFound desc = could not find container \"dea925b60400ef74c5ae2ac44ef5e9c36389eee54bfe7be7f6b182bd301b15ff\": container with ID starting with dea925b60400ef74c5ae2ac44ef5e9c36389eee54bfe7be7f6b182bd301b15ff not found: ID does not exist"
Jul 07 17:10:53.920965 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.920919    2579 scope.go:117] "RemoveContainer" containerID="a2f59130db609673a191b34a796cbbf6fcd149a8545a58f1cd6c39ad74dbbab3"
Jul 07 17:10:53.921114 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:10:53.921098    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2f59130db609673a191b34a796cbbf6fcd149a8545a58f1cd6c39ad74dbbab3\": container with ID starting with a2f59130db609673a191b34a796cbbf6fcd149a8545a58f1cd6c39ad74dbbab3 not found: ID does not exist" containerID="a2f59130db609673a191b34a796cbbf6fcd149a8545a58f1cd6c39ad74dbbab3"
Jul 07 17:10:53.921168 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.921116    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2f59130db609673a191b34a796cbbf6fcd149a8545a58f1cd6c39ad74dbbab3"} err="failed to get container status \"a2f59130db609673a191b34a796cbbf6fcd149a8545a58f1cd6c39ad74dbbab3\": rpc error: code = NotFound desc = could not find container \"a2f59130db609673a191b34a796cbbf6fcd149a8545a58f1cd6c39ad74dbbab3\": container with ID starting with a2f59130db609673a191b34a796cbbf6fcd149a8545a58f1cd6c39ad74dbbab3 not found: ID does not exist"
Jul 07 17:10:53.921168 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.921127    2579 scope.go:117] "RemoveContainer" containerID="e3f085ed8401f7f5efbed6861d5b8fd8149cf59ca5c4a83a988adccf7b2897f2"
Jul 07 17:10:53.921368 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:10:53.921351    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3f085ed8401f7f5efbed6861d5b8fd8149cf59ca5c4a83a988adccf7b2897f2\": container with ID starting with e3f085ed8401f7f5efbed6861d5b8fd8149cf59ca5c4a83a988adccf7b2897f2 not found: ID does not exist" containerID="e3f085ed8401f7f5efbed6861d5b8fd8149cf59ca5c4a83a988adccf7b2897f2"
Jul 07 17:10:53.921409 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.921373    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3f085ed8401f7f5efbed6861d5b8fd8149cf59ca5c4a83a988adccf7b2897f2"} err="failed to get container status \"e3f085ed8401f7f5efbed6861d5b8fd8149cf59ca5c4a83a988adccf7b2897f2\": rpc error: code = NotFound desc = could not find container \"e3f085ed8401f7f5efbed6861d5b8fd8149cf59ca5c4a83a988adccf7b2897f2\": container with ID starting with e3f085ed8401f7f5efbed6861d5b8fd8149cf59ca5c4a83a988adccf7b2897f2 not found: ID does not exist"
Jul 07 17:10:53.921409 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.921389    2579 scope.go:117] "RemoveContainer" containerID="c83b3b2ffa0b7b65512995b8a5aa57e65cf0a7d3d34b5985521295c48c21b0ff"
Jul 07 17:10:53.921669 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.921642    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c83b3b2ffa0b7b65512995b8a5aa57e65cf0a7d3d34b5985521295c48c21b0ff"} err="failed to get container status \"c83b3b2ffa0b7b65512995b8a5aa57e65cf0a7d3d34b5985521295c48c21b0ff\": rpc error: code = NotFound desc = could not find container \"c83b3b2ffa0b7b65512995b8a5aa57e65cf0a7d3d34b5985521295c48c21b0ff\": container with ID starting with c83b3b2ffa0b7b65512995b8a5aa57e65cf0a7d3d34b5985521295c48c21b0ff not found: ID does not exist"
Jul 07 17:10:53.921669 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.921668    2579 scope.go:117] "RemoveContainer" containerID="91c18870f3293b4d92ee1afd07ff140e4f3b37b28e8fc2086e11ffbd27a23a81"
Jul 07 17:10:53.921925 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.921860    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91c18870f3293b4d92ee1afd07ff140e4f3b37b28e8fc2086e11ffbd27a23a81"} err="failed to get container status \"91c18870f3293b4d92ee1afd07ff140e4f3b37b28e8fc2086e11ffbd27a23a81\": rpc error: code = NotFound desc = could not find container \"91c18870f3293b4d92ee1afd07ff140e4f3b37b28e8fc2086e11ffbd27a23a81\": container with ID starting with 91c18870f3293b4d92ee1afd07ff140e4f3b37b28e8fc2086e11ffbd27a23a81 not found: ID does not exist"
Jul 07 17:10:53.921925 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.921908    2579 scope.go:117] "RemoveContainer" containerID="86c2bc272b350716d0dac9ebc062e07ea3a24178f1b988a80196191e804e0456"
Jul 07 17:10:53.922146 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.922125    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86c2bc272b350716d0dac9ebc062e07ea3a24178f1b988a80196191e804e0456"} err="failed to get container status \"86c2bc272b350716d0dac9ebc062e07ea3a24178f1b988a80196191e804e0456\": rpc error: code = NotFound desc = could not find container \"86c2bc272b350716d0dac9ebc062e07ea3a24178f1b988a80196191e804e0456\": container with ID starting with 86c2bc272b350716d0dac9ebc062e07ea3a24178f1b988a80196191e804e0456 not found: ID does not exist"
Jul 07 17:10:53.922204 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.922146    2579 scope.go:117] "RemoveContainer" containerID="9159ca0524ded607b6e0661d3d8a603b5713a5ddfe84b7a483750666e01cebfd"
Jul 07 17:10:53.922375 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.922356    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9159ca0524ded607b6e0661d3d8a603b5713a5ddfe84b7a483750666e01cebfd"} err="failed to get container status \"9159ca0524ded607b6e0661d3d8a603b5713a5ddfe84b7a483750666e01cebfd\": rpc error: code = NotFound desc = could not find container \"9159ca0524ded607b6e0661d3d8a603b5713a5ddfe84b7a483750666e01cebfd\": container with ID starting with 9159ca0524ded607b6e0661d3d8a603b5713a5ddfe84b7a483750666e01cebfd not found: ID does not exist"
Jul 07 17:10:53.922448 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.922376    2579 scope.go:117] "RemoveContainer" containerID="dea925b60400ef74c5ae2ac44ef5e9c36389eee54bfe7be7f6b182bd301b15ff"
Jul 07 17:10:53.922604 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.922579    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dea925b60400ef74c5ae2ac44ef5e9c36389eee54bfe7be7f6b182bd301b15ff"} err="failed to get container status \"dea925b60400ef74c5ae2ac44ef5e9c36389eee54bfe7be7f6b182bd301b15ff\": rpc error: code = NotFound desc = could not find container \"dea925b60400ef74c5ae2ac44ef5e9c36389eee54bfe7be7f6b182bd301b15ff\": container with ID starting with dea925b60400ef74c5ae2ac44ef5e9c36389eee54bfe7be7f6b182bd301b15ff not found: ID does not exist"
Jul 07 17:10:53.922604 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.922598    2579 scope.go:117] "RemoveContainer" containerID="a2f59130db609673a191b34a796cbbf6fcd149a8545a58f1cd6c39ad74dbbab3"
Jul 07 17:10:53.922810 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.922789    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2f59130db609673a191b34a796cbbf6fcd149a8545a58f1cd6c39ad74dbbab3"} err="failed to get container status \"a2f59130db609673a191b34a796cbbf6fcd149a8545a58f1cd6c39ad74dbbab3\": rpc error: code = NotFound desc = could not find container \"a2f59130db609673a191b34a796cbbf6fcd149a8545a58f1cd6c39ad74dbbab3\": container with ID starting with a2f59130db609673a191b34a796cbbf6fcd149a8545a58f1cd6c39ad74dbbab3 not found: ID does not exist"
Jul 07 17:10:53.922848 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.922811    2579 scope.go:117] "RemoveContainer" containerID="e3f085ed8401f7f5efbed6861d5b8fd8149cf59ca5c4a83a988adccf7b2897f2"
Jul 07 17:10:53.923042 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.923027    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3f085ed8401f7f5efbed6861d5b8fd8149cf59ca5c4a83a988adccf7b2897f2"} err="failed to get container status \"e3f085ed8401f7f5efbed6861d5b8fd8149cf59ca5c4a83a988adccf7b2897f2\": rpc error: code = NotFound desc = could not find container \"e3f085ed8401f7f5efbed6861d5b8fd8149cf59ca5c4a83a988adccf7b2897f2\": container with ID starting with e3f085ed8401f7f5efbed6861d5b8fd8149cf59ca5c4a83a988adccf7b2897f2 not found: ID does not exist"
Jul 07 17:10:53.949931 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.949906    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 07 17:10:53.953386 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.953363    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:10:53.956009 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.955807    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 07 17:10:53.956009 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.955874    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 07 17:10:53.956157 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.956067    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 07 17:10:53.956223 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.956187    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 07 17:10:53.956223 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.956184    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 07 17:10:53.956223 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.956214    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 07 17:10:53.956396 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.956379    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 07 17:10:53.956506 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.956412    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 07 17:10:53.956560 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.956504    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-df5fn\""
Jul 07 17:10:53.960802 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.960783    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 07 17:10:53.969807 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:53.969788    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 07 17:10:54.004026 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:54.003992    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:10:54.004183 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:54.004031    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:10:54.004183 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:54.004065    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61-web-config\") pod \"alertmanager-main-0\" (UID: \"1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:10:54.004183 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:54.004087    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:10:54.004183 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:54.004113    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:10:54.004183 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:54.004137    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:10:54.004183 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:54.004161    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:10:54.004395 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:54.004230    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61-config-volume\") pod \"alertmanager-main-0\" (UID: \"1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:10:54.004395 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:54.004255    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61-config-out\") pod \"alertmanager-main-0\" (UID: \"1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:10:54.004395 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:54.004279    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:10:54.004395 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:54.004301    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m542v\" (UniqueName: \"kubernetes.io/projected/1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61-kube-api-access-m542v\") pod \"alertmanager-main-0\" (UID: \"1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:10:54.004395 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:54.004347    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:10:54.004395 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:54.004386    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:10:54.105226 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:54.105186    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:10:54.105226 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:54.105225    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61-web-config\") pod \"alertmanager-main-0\" (UID: \"1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:10:54.105466 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:54.105244    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:10:54.105466 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:54.105262    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:10:54.105466 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:54.105401    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:10:54.105466 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:54.105436    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:10:54.105652 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:54.105499    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61-config-volume\") pod \"alertmanager-main-0\" (UID: \"1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:10:54.105652 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:54.105522    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61-config-out\") pod \"alertmanager-main-0\" (UID: \"1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:10:54.105652 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:54.105556    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:10:54.105652 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:54.105604    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m542v\" (UniqueName: \"kubernetes.io/projected/1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61-kube-api-access-m542v\") pod \"alertmanager-main-0\" (UID: \"1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:10:54.105652 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:54.105638    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:10:54.105929 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:54.105679    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:10:54.105929 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:54.105718    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:10:54.105929 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:54.105759    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:10:54.106249 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:54.106209    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:10:54.108585 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:54.108553    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:10:54.108716 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:54.108639    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:10:54.108716 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:54.108661    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61-web-config\") pod \"alertmanager-main-0\" (UID: \"1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:10:54.108830 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:54.108747    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61-config-volume\") pod \"alertmanager-main-0\" (UID: \"1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:10:54.109018 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:54.108970    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:10:54.109129 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:54.109105    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:10:54.109243 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:54.109216    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:10:54.109425 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:54.109405    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:10:54.109528 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:54.109508    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:10:54.110472 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:54.110451    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61-config-out\") pod \"alertmanager-main-0\" (UID: \"1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:10:54.114930 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:54.114909    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m542v\" (UniqueName: \"kubernetes.io/projected/1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61-kube-api-access-m542v\") pod \"alertmanager-main-0\" (UID: \"1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:10:54.263608 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:54.263517    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 07 17:10:54.400777 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:54.400632    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 07 17:10:54.403008 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:10:54.402972    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ec998c7_6f2c_4961_8c48_6ad5dcf5bd61.slice/crio-53935da332fe133d332056d321b8e6c2f07c789f1089c526a7beb5d6702895d1 WatchSource:0}: Error finding container 53935da332fe133d332056d321b8e6c2f07c789f1089c526a7beb5d6702895d1: Status 404 returned error can't find the container with id 53935da332fe133d332056d321b8e6c2f07c789f1089c526a7beb5d6702895d1
Jul 07 17:10:54.872715 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:54.872679    2579 generic.go:356] "Generic (PLEG): container finished" podID="1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61" containerID="242840daa0069f839b4cdf9f19c3f40cd553b6c34c40795874773b2fe676fc7c" exitCode=0
Jul 07 17:10:54.872858 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:54.872776    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61","Type":"ContainerDied","Data":"242840daa0069f839b4cdf9f19c3f40cd553b6c34c40795874773b2fe676fc7c"}
Jul 07 17:10:54.872858 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:54.872835    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61","Type":"ContainerStarted","Data":"53935da332fe133d332056d321b8e6c2f07c789f1089c526a7beb5d6702895d1"}
Jul 07 17:10:55.027609 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:55.027576    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ace5af06-9cac-4a3c-826a-c4a33dcbed95" path="/var/lib/kubelet/pods/ace5af06-9cac-4a3c-826a-c4a33dcbed95/volumes"
Jul 07 17:10:55.879117 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:55.879081    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61","Type":"ContainerStarted","Data":"6983a8edb8b85a7d2a1ae02952109ddcf78bae27aa6a807622c5868c4be8743d"}
Jul 07 17:10:55.879531 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:55.879146    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61","Type":"ContainerStarted","Data":"8a033e1fb04444cada52f37fb96c13b29afeaffd5d715d2e75ec7a491c7f4053"}
Jul 07 17:10:55.879531 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:55.879173    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61","Type":"ContainerStarted","Data":"450f45c50ce9a7fa6b7eba6d49b43fd21e9294c939df2a20c072f13c113b5a12"}
Jul 07 17:10:55.879531 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:55.879209    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61","Type":"ContainerStarted","Data":"3dcf823d3eee985edf7424d4925bf702858ad522995bccdbc913a5d8efe5ad77"}
Jul 07 17:10:55.879531 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:55.879236    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61","Type":"ContainerStarted","Data":"bfabb259912ee7c47cf1b2d001ee7dda18d17a8c368ba3a5fd38fcb2f832928a"}
Jul 07 17:10:55.879531 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:55.879261    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61","Type":"ContainerStarted","Data":"341d87f36a20aece99da03513422316d50c09789ebb40597f701970a2d76876c"}
Jul 07 17:10:55.909937 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:10:55.909865    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.895278218 podStartE2EDuration="2.909847673s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.569455ms" isStatefulPod=false podCreationTimestamp="2026-07-07 17:10:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:10:55.907466274 +0000 UTC m=+179.403544782" watchObservedRunningTime="2026-07-07 17:10:55.909847673 +0000 UTC m=+179.405926208"
Jul 07 17:11:03.650288 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:11:03.650248    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-dd4f44b6d-khn84"
Jul 07 17:11:03.650769 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:11:03.650301    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-dd4f44b6d-khn84"
Jul 07 17:11:03.654846 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:11:03.654826    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-dd4f44b6d-khn84"
Jul 07 17:11:03.912659 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:11:03.912589    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-dd4f44b6d-khn84"
Jul 07 17:11:56.534483 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:11:56.534452    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m77pz"]
Jul 07 17:11:56.536766 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:11:56.536750    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m77pz"
Jul 07 17:11:56.539460 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:11:56.539443    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 07 17:11:56.540060 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:11:56.540042    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 07 17:11:56.540120 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:11:56.540042    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-pmnt5\""
Jul 07 17:11:56.547079 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:11:56.547056    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m77pz"]
Jul 07 17:11:56.646069 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:11:56.646040    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4468fe47-8687-41bb-9e65-845f178e3568-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m77pz\" (UID: \"4468fe47-8687-41bb-9e65-845f178e3568\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m77pz"
Jul 07 17:11:56.646189 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:11:56.646087    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckgwq\" (UniqueName: \"kubernetes.io/projected/4468fe47-8687-41bb-9e65-845f178e3568-kube-api-access-ckgwq\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m77pz\" (UID: \"4468fe47-8687-41bb-9e65-845f178e3568\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m77pz"
Jul 07 17:11:56.646189 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:11:56.646149    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4468fe47-8687-41bb-9e65-845f178e3568-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m77pz\" (UID: \"4468fe47-8687-41bb-9e65-845f178e3568\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m77pz"
Jul 07 17:11:56.747164 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:11:56.747142    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4468fe47-8687-41bb-9e65-845f178e3568-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m77pz\" (UID: \"4468fe47-8687-41bb-9e65-845f178e3568\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m77pz"
Jul 07 17:11:56.747280 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:11:56.747186    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckgwq\" (UniqueName: \"kubernetes.io/projected/4468fe47-8687-41bb-9e65-845f178e3568-kube-api-access-ckgwq\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m77pz\" (UID: \"4468fe47-8687-41bb-9e65-845f178e3568\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m77pz"
Jul 07 17:11:56.747280 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:11:56.747218    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4468fe47-8687-41bb-9e65-845f178e3568-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m77pz\" (UID: \"4468fe47-8687-41bb-9e65-845f178e3568\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m77pz"
Jul 07 17:11:56.747582 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:11:56.747562    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4468fe47-8687-41bb-9e65-845f178e3568-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m77pz\" (UID: \"4468fe47-8687-41bb-9e65-845f178e3568\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m77pz"
Jul 07 17:11:56.747648 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:11:56.747576    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4468fe47-8687-41bb-9e65-845f178e3568-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m77pz\" (UID: \"4468fe47-8687-41bb-9e65-845f178e3568\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m77pz"
Jul 07 17:11:56.757074 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:11:56.757053    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckgwq\" (UniqueName: \"kubernetes.io/projected/4468fe47-8687-41bb-9e65-845f178e3568-kube-api-access-ckgwq\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m77pz\" (UID: \"4468fe47-8687-41bb-9e65-845f178e3568\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m77pz"
Jul 07 17:11:56.848536 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:11:56.848482    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-pmnt5\""
Jul 07 17:11:56.856613 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:11:56.856596    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m77pz"
Jul 07 17:11:56.986917 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:11:56.986874    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m77pz"]
Jul 07 17:11:56.990139 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:11:56.990113    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4468fe47_8687_41bb_9e65_845f178e3568.slice/crio-07f2c7a95a4a03833284b197b7c51802742f0f9cd5e596568d8e8db33fbbffa8 WatchSource:0}: Error finding container 07f2c7a95a4a03833284b197b7c51802742f0f9cd5e596568d8e8db33fbbffa8: Status 404 returned error can't find the container with id 07f2c7a95a4a03833284b197b7c51802742f0f9cd5e596568d8e8db33fbbffa8
Jul 07 17:11:57.090101 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:11:57.090072    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m77pz" event={"ID":"4468fe47-8687-41bb-9e65-845f178e3568","Type":"ContainerStarted","Data":"07f2c7a95a4a03833284b197b7c51802742f0f9cd5e596568d8e8db33fbbffa8"}
Jul 07 17:12:07.129905 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:07.129855    2579 generic.go:356] "Generic (PLEG): container finished" podID="4468fe47-8687-41bb-9e65-845f178e3568" containerID="84fccaf71a5daff653473a6dfddb4dbb65d5f09cb0374661efe2bc9fd2f8f00d" exitCode=0
Jul 07 17:12:07.130269 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:07.129924    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m77pz" event={"ID":"4468fe47-8687-41bb-9e65-845f178e3568","Type":"ContainerDied","Data":"84fccaf71a5daff653473a6dfddb4dbb65d5f09cb0374661efe2bc9fd2f8f00d"}
Jul 07 17:12:09.138763 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:09.138674    2579 generic.go:356] "Generic (PLEG): container finished" podID="4468fe47-8687-41bb-9e65-845f178e3568" containerID="fcd7a92606472ef0b974d5e68df8da6aa39fcf4373c20afd476e4fdbc0919556" exitCode=0
Jul 07 17:12:09.138763 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:09.138719    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m77pz" event={"ID":"4468fe47-8687-41bb-9e65-845f178e3568","Type":"ContainerDied","Data":"fcd7a92606472ef0b974d5e68df8da6aa39fcf4373c20afd476e4fdbc0919556"}
Jul 07 17:12:20.183477 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:20.183449    2579 generic.go:356] "Generic (PLEG): container finished" podID="4468fe47-8687-41bb-9e65-845f178e3568" containerID="46ee6c9c5c6b4a17fd7e44b2963b3798efefd2cc59f0bc23d34509b61e2a5fc1" exitCode=0
Jul 07 17:12:20.183909 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:20.183531    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m77pz" event={"ID":"4468fe47-8687-41bb-9e65-845f178e3568","Type":"ContainerDied","Data":"46ee6c9c5c6b4a17fd7e44b2963b3798efefd2cc59f0bc23d34509b61e2a5fc1"}
Jul 07 17:12:21.315009 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:21.314985    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m77pz"
Jul 07 17:12:21.440705 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:21.440653    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4468fe47-8687-41bb-9e65-845f178e3568-bundle\") pod \"4468fe47-8687-41bb-9e65-845f178e3568\" (UID: \"4468fe47-8687-41bb-9e65-845f178e3568\") "
Jul 07 17:12:21.440705 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:21.440698    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ckgwq\" (UniqueName: \"kubernetes.io/projected/4468fe47-8687-41bb-9e65-845f178e3568-kube-api-access-ckgwq\") pod \"4468fe47-8687-41bb-9e65-845f178e3568\" (UID: \"4468fe47-8687-41bb-9e65-845f178e3568\") "
Jul 07 17:12:21.440836 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:21.440732    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4468fe47-8687-41bb-9e65-845f178e3568-util\") pod \"4468fe47-8687-41bb-9e65-845f178e3568\" (UID: \"4468fe47-8687-41bb-9e65-845f178e3568\") "
Jul 07 17:12:21.441302 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:21.441271    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4468fe47-8687-41bb-9e65-845f178e3568-bundle" (OuterVolumeSpecName: "bundle") pod "4468fe47-8687-41bb-9e65-845f178e3568" (UID: "4468fe47-8687-41bb-9e65-845f178e3568"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:12:21.442910 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:21.442867    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4468fe47-8687-41bb-9e65-845f178e3568-kube-api-access-ckgwq" (OuterVolumeSpecName: "kube-api-access-ckgwq") pod "4468fe47-8687-41bb-9e65-845f178e3568" (UID: "4468fe47-8687-41bb-9e65-845f178e3568"). InnerVolumeSpecName "kube-api-access-ckgwq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 17:12:21.444852 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:21.444834    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4468fe47-8687-41bb-9e65-845f178e3568-util" (OuterVolumeSpecName: "util") pod "4468fe47-8687-41bb-9e65-845f178e3568" (UID: "4468fe47-8687-41bb-9e65-845f178e3568"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:12:21.541536 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:21.541514    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ckgwq\" (UniqueName: \"kubernetes.io/projected/4468fe47-8687-41bb-9e65-845f178e3568-kube-api-access-ckgwq\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:12:21.541536 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:21.541534    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4468fe47-8687-41bb-9e65-845f178e3568-util\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:12:21.541653 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:21.541544    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4468fe47-8687-41bb-9e65-845f178e3568-bundle\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:12:22.191221 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:22.191196    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m77pz"
Jul 07 17:12:22.191221 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:22.191206    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m77pz" event={"ID":"4468fe47-8687-41bb-9e65-845f178e3568","Type":"ContainerDied","Data":"07f2c7a95a4a03833284b197b7c51802742f0f9cd5e596568d8e8db33fbbffa8"}
Jul 07 17:12:22.191436 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:22.191240    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="07f2c7a95a4a03833284b197b7c51802742f0f9cd5e596568d8e8db33fbbffa8"
Jul 07 17:12:29.823342 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:29.823309    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-98l59"]
Jul 07 17:12:29.823707 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:29.823575    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4468fe47-8687-41bb-9e65-845f178e3568" containerName="pull"
Jul 07 17:12:29.823707 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:29.823586    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="4468fe47-8687-41bb-9e65-845f178e3568" containerName="pull"
Jul 07 17:12:29.823707 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:29.823593    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4468fe47-8687-41bb-9e65-845f178e3568" containerName="util"
Jul 07 17:12:29.823707 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:29.823598    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="4468fe47-8687-41bb-9e65-845f178e3568" containerName="util"
Jul 07 17:12:29.823707 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:29.823608    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4468fe47-8687-41bb-9e65-845f178e3568" containerName="extract"
Jul 07 17:12:29.823707 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:29.823613    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="4468fe47-8687-41bb-9e65-845f178e3568" containerName="extract"
Jul 07 17:12:29.823707 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:29.823665    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="4468fe47-8687-41bb-9e65-845f178e3568" containerName="extract"
Jul 07 17:12:29.827144 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:29.827128    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-98l59"
Jul 07 17:12:29.829499 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:29.829478    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 07 17:12:29.829625 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:29.829503    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 07 17:12:29.829919 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:29.829902    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-ccxjl\""
Jul 07 17:12:29.838795 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:29.838775    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-98l59"]
Jul 07 17:12:29.902867 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:29.902838    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwdf9\" (UniqueName: \"kubernetes.io/projected/f79e9c2b-e9cb-4abb-834f-f0c3f8830280-kube-api-access-pwdf9\") pod \"cert-manager-operator-controller-manager-55c444bf86-98l59\" (UID: \"f79e9c2b-e9cb-4abb-834f-f0c3f8830280\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-98l59"
Jul 07 17:12:29.902973 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:29.902954    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f79e9c2b-e9cb-4abb-834f-f0c3f8830280-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-98l59\" (UID: \"f79e9c2b-e9cb-4abb-834f-f0c3f8830280\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-98l59"
Jul 07 17:12:30.003594 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:30.003558    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f79e9c2b-e9cb-4abb-834f-f0c3f8830280-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-98l59\" (UID: \"f79e9c2b-e9cb-4abb-834f-f0c3f8830280\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-98l59"
Jul 07 17:12:30.003697 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:30.003613    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwdf9\" (UniqueName: \"kubernetes.io/projected/f79e9c2b-e9cb-4abb-834f-f0c3f8830280-kube-api-access-pwdf9\") pod \"cert-manager-operator-controller-manager-55c444bf86-98l59\" (UID: \"f79e9c2b-e9cb-4abb-834f-f0c3f8830280\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-98l59"
Jul 07 17:12:30.003933 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:30.003916    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f79e9c2b-e9cb-4abb-834f-f0c3f8830280-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-98l59\" (UID: \"f79e9c2b-e9cb-4abb-834f-f0c3f8830280\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-98l59"
Jul 07 17:12:30.013520 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:30.013499    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwdf9\" (UniqueName: \"kubernetes.io/projected/f79e9c2b-e9cb-4abb-834f-f0c3f8830280-kube-api-access-pwdf9\") pod \"cert-manager-operator-controller-manager-55c444bf86-98l59\" (UID: \"f79e9c2b-e9cb-4abb-834f-f0c3f8830280\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-98l59"
Jul 07 17:12:30.135621 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:30.135572    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-98l59"
Jul 07 17:12:30.280179 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:30.280139    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-98l59"]
Jul 07 17:12:30.284364 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:12:30.284340    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf79e9c2b_e9cb_4abb_834f_f0c3f8830280.slice/crio-b63bb246419ab6875e9afe1856c0d2c267d890a844d39bc9ce5d23de71d4a217 WatchSource:0}: Error finding container b63bb246419ab6875e9afe1856c0d2c267d890a844d39bc9ce5d23de71d4a217: Status 404 returned error can't find the container with id b63bb246419ab6875e9afe1856c0d2c267d890a844d39bc9ce5d23de71d4a217
Jul 07 17:12:31.236981 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:31.236944    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-98l59" event={"ID":"f79e9c2b-e9cb-4abb-834f-f0c3f8830280","Type":"ContainerStarted","Data":"b63bb246419ab6875e9afe1856c0d2c267d890a844d39bc9ce5d23de71d4a217"}
Jul 07 17:12:32.242535 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:32.242458    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-98l59" event={"ID":"f79e9c2b-e9cb-4abb-834f-f0c3f8830280","Type":"ContainerStarted","Data":"05a6dc53d0066b66b252ab173238f7cd2ac8fc6aac739ceedfb1cb156764da2b"}
Jul 07 17:12:32.267315 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:32.267273    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-98l59" podStartSLOduration=1.590490867 podStartE2EDuration="3.267259146s" totalImagesPullingTime="1.676768279s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:12:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:12:32.266238007 +0000 UTC m=+275.762316514" watchObservedRunningTime="2026-07-07 17:12:32.267259146 +0000 UTC m=+275.763337654"
Jul 07 17:12:33.516701 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:33.516668    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flfldq"]
Jul 07 17:12:33.519014 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:33.518998    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flfldq"
Jul 07 17:12:33.521298 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:33.521278    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 07 17:12:33.521419 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:33.521333    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-pmnt5\""
Jul 07 17:12:33.521419 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:33.521332    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 07 17:12:33.532093 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:33.532073    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flfldq"]
Jul 07 17:12:33.629500 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:33.629468    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8d6kr\" (UniqueName: \"kubernetes.io/projected/521f6043-c4c8-4349-9b26-aced0660f6b3-kube-api-access-8d6kr\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flfldq\" (UID: \"521f6043-c4c8-4349-9b26-aced0660f6b3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flfldq"
Jul 07 17:12:33.629638 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:33.629533    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/521f6043-c4c8-4349-9b26-aced0660f6b3-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flfldq\" (UID: \"521f6043-c4c8-4349-9b26-aced0660f6b3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flfldq"
Jul 07 17:12:33.629638 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:33.629557    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/521f6043-c4c8-4349-9b26-aced0660f6b3-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flfldq\" (UID: \"521f6043-c4c8-4349-9b26-aced0660f6b3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flfldq"
Jul 07 17:12:33.730138 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:33.730106    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/521f6043-c4c8-4349-9b26-aced0660f6b3-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flfldq\" (UID: \"521f6043-c4c8-4349-9b26-aced0660f6b3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flfldq"
Jul 07 17:12:33.730282 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:33.730140    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/521f6043-c4c8-4349-9b26-aced0660f6b3-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flfldq\" (UID: \"521f6043-c4c8-4349-9b26-aced0660f6b3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flfldq"
Jul 07 17:12:33.730282 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:33.730198    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8d6kr\" (UniqueName: \"kubernetes.io/projected/521f6043-c4c8-4349-9b26-aced0660f6b3-kube-api-access-8d6kr\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flfldq\" (UID: \"521f6043-c4c8-4349-9b26-aced0660f6b3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flfldq"
Jul 07 17:12:33.730557 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:33.730537    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/521f6043-c4c8-4349-9b26-aced0660f6b3-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flfldq\" (UID: \"521f6043-c4c8-4349-9b26-aced0660f6b3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flfldq"
Jul 07 17:12:33.730623 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:33.730561    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/521f6043-c4c8-4349-9b26-aced0660f6b3-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flfldq\" (UID: \"521f6043-c4c8-4349-9b26-aced0660f6b3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flfldq"
Jul 07 17:12:33.744239 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:33.744216    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8d6kr\" (UniqueName: \"kubernetes.io/projected/521f6043-c4c8-4349-9b26-aced0660f6b3-kube-api-access-8d6kr\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flfldq\" (UID: \"521f6043-c4c8-4349-9b26-aced0660f6b3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flfldq"
Jul 07 17:12:33.827945 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:33.827836    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flfldq"
Jul 07 17:12:33.966034 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:33.966011    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flfldq"]
Jul 07 17:12:33.968688 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:12:33.968662    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod521f6043_c4c8_4349_9b26_aced0660f6b3.slice/crio-1986bdf37b53ea20128c22e4384927667ef46160749c5d6b16fc8f65ac7cbdfa WatchSource:0}: Error finding container 1986bdf37b53ea20128c22e4384927667ef46160749c5d6b16fc8f65ac7cbdfa: Status 404 returned error can't find the container with id 1986bdf37b53ea20128c22e4384927667ef46160749c5d6b16fc8f65ac7cbdfa
Jul 07 17:12:34.252201 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:34.252174    2579 generic.go:356] "Generic (PLEG): container finished" podID="521f6043-c4c8-4349-9b26-aced0660f6b3" containerID="48642a40c458c392eb6ae0d38c6fb1c2b77988bec81d04226de383dcb000bba7" exitCode=0
Jul 07 17:12:34.252313 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:34.252231    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flfldq" event={"ID":"521f6043-c4c8-4349-9b26-aced0660f6b3","Type":"ContainerDied","Data":"48642a40c458c392eb6ae0d38c6fb1c2b77988bec81d04226de383dcb000bba7"}
Jul 07 17:12:34.252313 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:34.252270    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flfldq" event={"ID":"521f6043-c4c8-4349-9b26-aced0660f6b3","Type":"ContainerStarted","Data":"1986bdf37b53ea20128c22e4384927667ef46160749c5d6b16fc8f65ac7cbdfa"}
Jul 07 17:12:36.535905 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:36.535856    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-x85z7"]
Jul 07 17:12:36.537809 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:36.537795    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-x85z7"
Jul 07 17:12:36.539803 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:36.539784    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 07 17:12:36.540556 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:36.540543    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 07 17:12:36.540603 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:36.540590    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-b62p4\""
Jul 07 17:12:36.554728 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:36.554705    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-x85z7"]
Jul 07 17:12:36.654769 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:36.654747    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/605418ea-724d-4fd0-9d0c-662039088c04-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-x85z7\" (UID: \"605418ea-724d-4fd0-9d0c-662039088c04\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-x85z7"
Jul 07 17:12:36.654864 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:36.654783    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpjkp\" (UniqueName: \"kubernetes.io/projected/605418ea-724d-4fd0-9d0c-662039088c04-kube-api-access-zpjkp\") pod \"cert-manager-cainjector-559cdd8c87-x85z7\" (UID: \"605418ea-724d-4fd0-9d0c-662039088c04\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-x85z7"
Jul 07 17:12:36.755257 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:36.755230    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/605418ea-724d-4fd0-9d0c-662039088c04-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-x85z7\" (UID: \"605418ea-724d-4fd0-9d0c-662039088c04\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-x85z7"
Jul 07 17:12:36.755338 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:36.755269    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpjkp\" (UniqueName: \"kubernetes.io/projected/605418ea-724d-4fd0-9d0c-662039088c04-kube-api-access-zpjkp\") pod \"cert-manager-cainjector-559cdd8c87-x85z7\" (UID: \"605418ea-724d-4fd0-9d0c-662039088c04\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-x85z7"
Jul 07 17:12:36.765397 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:36.765368    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpjkp\" (UniqueName: \"kubernetes.io/projected/605418ea-724d-4fd0-9d0c-662039088c04-kube-api-access-zpjkp\") pod \"cert-manager-cainjector-559cdd8c87-x85z7\" (UID: \"605418ea-724d-4fd0-9d0c-662039088c04\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-x85z7"
Jul 07 17:12:36.765471 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:36.765409    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/605418ea-724d-4fd0-9d0c-662039088c04-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-x85z7\" (UID: \"605418ea-724d-4fd0-9d0c-662039088c04\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-x85z7"
Jul 07 17:12:36.846544 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:36.846526    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-x85z7"
Jul 07 17:12:36.968571 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:36.968548    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-x85z7"]
Jul 07 17:12:36.970650 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:12:36.970626    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod605418ea_724d_4fd0_9d0c_662039088c04.slice/crio-c2f68192db9cf0963042c40ce54a294a296425a8adfad51812b4b2a84c301f33 WatchSource:0}: Error finding container c2f68192db9cf0963042c40ce54a294a296425a8adfad51812b4b2a84c301f33: Status 404 returned error can't find the container with id c2f68192db9cf0963042c40ce54a294a296425a8adfad51812b4b2a84c301f33
Jul 07 17:12:37.266067 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:37.265993    2579 generic.go:356] "Generic (PLEG): container finished" podID="521f6043-c4c8-4349-9b26-aced0660f6b3" containerID="6371cef7cfeb6336d4b30cd244764622db2839680f80aefbb5b04c1ef2553119" exitCode=0
Jul 07 17:12:37.266197 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:37.266078    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flfldq" event={"ID":"521f6043-c4c8-4349-9b26-aced0660f6b3","Type":"ContainerDied","Data":"6371cef7cfeb6336d4b30cd244764622db2839680f80aefbb5b04c1ef2553119"}
Jul 07 17:12:37.267277 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:37.267256    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-x85z7" event={"ID":"605418ea-724d-4fd0-9d0c-662039088c04","Type":"ContainerStarted","Data":"c2f68192db9cf0963042c40ce54a294a296425a8adfad51812b4b2a84c301f33"}
Jul 07 17:12:38.273975 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:38.273942    2579 generic.go:356] "Generic (PLEG): container finished" podID="521f6043-c4c8-4349-9b26-aced0660f6b3" containerID="fb4775588451e69d1042116edf59e35fe5557c0f91d86f0a3d30ef4fde68a033" exitCode=0
Jul 07 17:12:38.274415 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:38.273976    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flfldq" event={"ID":"521f6043-c4c8-4349-9b26-aced0660f6b3","Type":"ContainerDied","Data":"fb4775588451e69d1042116edf59e35fe5557c0f91d86f0a3d30ef4fde68a033"}
Jul 07 17:12:39.412189 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:39.412161    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flfldq"
Jul 07 17:12:39.477995 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:39.477966    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/521f6043-c4c8-4349-9b26-aced0660f6b3-util\") pod \"521f6043-c4c8-4349-9b26-aced0660f6b3\" (UID: \"521f6043-c4c8-4349-9b26-aced0660f6b3\") "
Jul 07 17:12:39.478166 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:39.478011    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/521f6043-c4c8-4349-9b26-aced0660f6b3-bundle\") pod \"521f6043-c4c8-4349-9b26-aced0660f6b3\" (UID: \"521f6043-c4c8-4349-9b26-aced0660f6b3\") "
Jul 07 17:12:39.478166 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:39.478077    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8d6kr\" (UniqueName: \"kubernetes.io/projected/521f6043-c4c8-4349-9b26-aced0660f6b3-kube-api-access-8d6kr\") pod \"521f6043-c4c8-4349-9b26-aced0660f6b3\" (UID: \"521f6043-c4c8-4349-9b26-aced0660f6b3\") "
Jul 07 17:12:39.478386 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:39.478368    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/521f6043-c4c8-4349-9b26-aced0660f6b3-bundle" (OuterVolumeSpecName: "bundle") pod "521f6043-c4c8-4349-9b26-aced0660f6b3" (UID: "521f6043-c4c8-4349-9b26-aced0660f6b3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:12:39.480254 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:39.480230    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/521f6043-c4c8-4349-9b26-aced0660f6b3-kube-api-access-8d6kr" (OuterVolumeSpecName: "kube-api-access-8d6kr") pod "521f6043-c4c8-4349-9b26-aced0660f6b3" (UID: "521f6043-c4c8-4349-9b26-aced0660f6b3"). InnerVolumeSpecName "kube-api-access-8d6kr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 17:12:39.547736 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:39.547667    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/521f6043-c4c8-4349-9b26-aced0660f6b3-util" (OuterVolumeSpecName: "util") pod "521f6043-c4c8-4349-9b26-aced0660f6b3" (UID: "521f6043-c4c8-4349-9b26-aced0660f6b3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:12:39.578900 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:39.578871    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/521f6043-c4c8-4349-9b26-aced0660f6b3-util\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:12:39.579078 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:39.578910    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/521f6043-c4c8-4349-9b26-aced0660f6b3-bundle\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:12:39.579078 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:39.578923    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8d6kr\" (UniqueName: \"kubernetes.io/projected/521f6043-c4c8-4349-9b26-aced0660f6b3-kube-api-access-8d6kr\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:12:40.283872 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:40.283837    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flfldq" event={"ID":"521f6043-c4c8-4349-9b26-aced0660f6b3","Type":"ContainerDied","Data":"1986bdf37b53ea20128c22e4384927667ef46160749c5d6b16fc8f65ac7cbdfa"}
Jul 07 17:12:40.284051 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:40.283847    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flfldq"
Jul 07 17:12:40.284051 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:40.283882    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1986bdf37b53ea20128c22e4384927667ef46160749c5d6b16fc8f65ac7cbdfa"
Jul 07 17:12:40.285529 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:40.285507    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-x85z7" event={"ID":"605418ea-724d-4fd0-9d0c-662039088c04","Type":"ContainerStarted","Data":"44cc5d3ea9bd366d2ea6411e07944de1c57b7585dcfc546c462a0b8c8924385c"}
Jul 07 17:12:40.304991 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:40.304954    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-x85z7" podStartSLOduration=1.983884234 podStartE2EDuration="4.304941226s" totalImagesPullingTime="2.321056992s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:12:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:12:40.304220365 +0000 UTC m=+283.800298876" watchObservedRunningTime="2026-07-07 17:12:40.304941226 +0000 UTC m=+283.801019731"
Jul 07 17:12:42.087163 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:42.087135    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-9zxmw"]
Jul 07 17:12:42.087513 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:42.087377    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="521f6043-c4c8-4349-9b26-aced0660f6b3" containerName="extract"
Jul 07 17:12:42.087513 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:42.087387    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="521f6043-c4c8-4349-9b26-aced0660f6b3" containerName="extract"
Jul 07 17:12:42.087513 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:42.087405    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="521f6043-c4c8-4349-9b26-aced0660f6b3" containerName="pull"
Jul 07 17:12:42.087513 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:42.087411    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="521f6043-c4c8-4349-9b26-aced0660f6b3" containerName="pull"
Jul 07 17:12:42.087513 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:42.087417    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="521f6043-c4c8-4349-9b26-aced0660f6b3" containerName="util"
Jul 07 17:12:42.087513 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:42.087422    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="521f6043-c4c8-4349-9b26-aced0660f6b3" containerName="util"
Jul 07 17:12:42.087513 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:42.087474    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="521f6043-c4c8-4349-9b26-aced0660f6b3" containerName="extract"
Jul 07 17:12:42.090301 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:42.090284    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-9zxmw"
Jul 07 17:12:42.092484 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:42.092455    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-n5gm5\""
Jul 07 17:12:42.101822 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:42.101801    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-9zxmw"]
Jul 07 17:12:42.194471 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:42.194447    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ea4955ff-b874-4465-a598-6265a15139db-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-9zxmw\" (UID: \"ea4955ff-b874-4465-a598-6265a15139db\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9zxmw"
Jul 07 17:12:42.194585 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:42.194508    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qcxs\" (UniqueName: \"kubernetes.io/projected/ea4955ff-b874-4465-a598-6265a15139db-kube-api-access-8qcxs\") pod \"cert-manager-webhook-6486ffd997-9zxmw\" (UID: \"ea4955ff-b874-4465-a598-6265a15139db\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9zxmw"
Jul 07 17:12:42.295515 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:42.295492    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qcxs\" (UniqueName: \"kubernetes.io/projected/ea4955ff-b874-4465-a598-6265a15139db-kube-api-access-8qcxs\") pod \"cert-manager-webhook-6486ffd997-9zxmw\" (UID: \"ea4955ff-b874-4465-a598-6265a15139db\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9zxmw"
Jul 07 17:12:42.295611 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:42.295534    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ea4955ff-b874-4465-a598-6265a15139db-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-9zxmw\" (UID: \"ea4955ff-b874-4465-a598-6265a15139db\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9zxmw"
Jul 07 17:12:42.309622 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:42.309597    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ea4955ff-b874-4465-a598-6265a15139db-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-9zxmw\" (UID: \"ea4955ff-b874-4465-a598-6265a15139db\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9zxmw"
Jul 07 17:12:42.309871 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:42.309849    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qcxs\" (UniqueName: \"kubernetes.io/projected/ea4955ff-b874-4465-a598-6265a15139db-kube-api-access-8qcxs\") pod \"cert-manager-webhook-6486ffd997-9zxmw\" (UID: \"ea4955ff-b874-4465-a598-6265a15139db\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9zxmw"
Jul 07 17:12:42.398314 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:42.398295    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-9zxmw"
Jul 07 17:12:42.528056 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:42.528029    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-9zxmw"]
Jul 07 17:12:42.530205 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:12:42.530179    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podea4955ff_b874_4465_a598_6265a15139db.slice/crio-d505aa3a321139fb4ec671f7aa7c6e093ecb26b16fd565d5ae6421c872c20f0d WatchSource:0}: Error finding container d505aa3a321139fb4ec671f7aa7c6e093ecb26b16fd565d5ae6421c872c20f0d: Status 404 returned error can't find the container with id d505aa3a321139fb4ec671f7aa7c6e093ecb26b16fd565d5ae6421c872c20f0d
Jul 07 17:12:43.298839 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:43.298806    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-9zxmw" event={"ID":"ea4955ff-b874-4465-a598-6265a15139db","Type":"ContainerStarted","Data":"0132ea8771b47a18e19fafced12d2816baee890e1b24040742cd345bd69777fd"}
Jul 07 17:12:43.299218 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:43.298849    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-9zxmw" event={"ID":"ea4955ff-b874-4465-a598-6265a15139db","Type":"ContainerStarted","Data":"d505aa3a321139fb4ec671f7aa7c6e093ecb26b16fd565d5ae6421c872c20f0d"}
Jul 07 17:12:43.299218 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:43.298941    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-9zxmw"
Jul 07 17:12:43.321504 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:43.321457    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-9zxmw" podStartSLOduration=1.321442727 podStartE2EDuration="1.321442727s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:12:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:12:43.320528892 +0000 UTC m=+286.816607400" watchObservedRunningTime="2026-07-07 17:12:43.321442727 +0000 UTC m=+286.817521235"
Jul 07 17:12:49.305490 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:49.305457    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-9zxmw"
Jul 07 17:12:54.833985 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:54.833952    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52w9l"]
Jul 07 17:12:54.838224 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:54.838208    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52w9l"
Jul 07 17:12:54.841256 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:54.841237    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 07 17:12:54.841875 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:54.841863    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-pmnt5\""
Jul 07 17:12:54.843110 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:54.843096    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 07 17:12:54.857578 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:54.857550    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52w9l"]
Jul 07 17:12:54.981783 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:54.981760    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52w9l\" (UID: \"a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52w9l"
Jul 07 17:12:54.981939 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:54.981804    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52w9l\" (UID: \"a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52w9l"
Jul 07 17:12:54.981939 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:54.981829    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrk5t\" (UniqueName: \"kubernetes.io/projected/a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8-kube-api-access-zrk5t\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52w9l\" (UID: \"a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52w9l"
Jul 07 17:12:55.082563 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:55.082538    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52w9l\" (UID: \"a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52w9l"
Jul 07 17:12:55.082684 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:55.082578    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52w9l\" (UID: \"a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52w9l"
Jul 07 17:12:55.082684 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:55.082608    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrk5t\" (UniqueName: \"kubernetes.io/projected/a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8-kube-api-access-zrk5t\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52w9l\" (UID: \"a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52w9l"
Jul 07 17:12:55.082941 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:55.082923    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52w9l\" (UID: \"a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52w9l"
Jul 07 17:12:55.082941 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:55.082934    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52w9l\" (UID: \"a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52w9l"
Jul 07 17:12:55.095287 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:55.095244    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrk5t\" (UniqueName: \"kubernetes.io/projected/a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8-kube-api-access-zrk5t\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52w9l\" (UID: \"a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52w9l"
Jul 07 17:12:55.146622 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:55.146597    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52w9l"
Jul 07 17:12:55.283287 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:55.283264    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52w9l"]
Jul 07 17:12:55.285708 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:12:55.285680    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3ccf3ee_d3e0_4848_802c_f1cd4d0501a8.slice/crio-c89c68995df1c10da5d7b0015075169e92727cf33d59b511010be5cd2de5f38e WatchSource:0}: Error finding container c89c68995df1c10da5d7b0015075169e92727cf33d59b511010be5cd2de5f38e: Status 404 returned error can't find the container with id c89c68995df1c10da5d7b0015075169e92727cf33d59b511010be5cd2de5f38e
Jul 07 17:12:55.347430 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:55.347380    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52w9l" event={"ID":"a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8","Type":"ContainerStarted","Data":"510f95a0f5cff93a8a524e738ebcc84657e8bcdd060cb2b8dca8e169515dcfeb"}
Jul 07 17:12:55.347526 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:55.347432    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52w9l" event={"ID":"a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8","Type":"ContainerStarted","Data":"c89c68995df1c10da5d7b0015075169e92727cf33d59b511010be5cd2de5f38e"}
Jul 07 17:12:56.352970 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:56.352882    2579 generic.go:356] "Generic (PLEG): container finished" podID="a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8" containerID="510f95a0f5cff93a8a524e738ebcc84657e8bcdd060cb2b8dca8e169515dcfeb" exitCode=0
Jul 07 17:12:56.352970 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:56.352923    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52w9l" event={"ID":"a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8","Type":"ContainerDied","Data":"510f95a0f5cff93a8a524e738ebcc84657e8bcdd060cb2b8dca8e169515dcfeb"}
Jul 07 17:12:56.890111 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:56.890087    2579 kubelet.go:1648] "Image garbage collection succeeded"
Jul 07 17:12:57.359257 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:57.359231    2579 generic.go:356] "Generic (PLEG): container finished" podID="a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8" containerID="d74e71fbf9898b7707260644b38a8a6e8cfd0423feea3ee5a422cf26f8e074d4" exitCode=0
Jul 07 17:12:57.359604 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:57.359270    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52w9l" event={"ID":"a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8","Type":"ContainerDied","Data":"d74e71fbf9898b7707260644b38a8a6e8cfd0423feea3ee5a422cf26f8e074d4"}
Jul 07 17:12:58.365800 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:58.365767    2579 generic.go:356] "Generic (PLEG): container finished" podID="a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8" containerID="f421676aef1bf118d111dd1cac5ac98a05e70313cb229a5e6a4990f7b15d08b7" exitCode=0
Jul 07 17:12:58.366191 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:58.365858    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52w9l" event={"ID":"a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8","Type":"ContainerDied","Data":"f421676aef1bf118d111dd1cac5ac98a05e70313cb229a5e6a4990f7b15d08b7"}
Jul 07 17:12:59.494873 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:59.494851    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52w9l"
Jul 07 17:12:59.614116 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:59.614088    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8-bundle\") pod \"a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8\" (UID: \"a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8\") "
Jul 07 17:12:59.614242 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:59.614123    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8-util\") pod \"a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8\" (UID: \"a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8\") "
Jul 07 17:12:59.614242 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:59.614177    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zrk5t\" (UniqueName: \"kubernetes.io/projected/a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8-kube-api-access-zrk5t\") pod \"a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8\" (UID: \"a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8\") "
Jul 07 17:12:59.614778 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:59.614749    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8-bundle" (OuterVolumeSpecName: "bundle") pod "a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8" (UID: "a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:12:59.616286 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:59.616255    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8-kube-api-access-zrk5t" (OuterVolumeSpecName: "kube-api-access-zrk5t") pod "a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8" (UID: "a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8"). InnerVolumeSpecName "kube-api-access-zrk5t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 17:12:59.619935 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:59.619917    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8-util" (OuterVolumeSpecName: "util") pod "a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8" (UID: "a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:12:59.715572 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:59.715548    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8-bundle\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:12:59.715572 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:59.715568    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8-util\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:12:59.715728 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:12:59.715576    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zrk5t\" (UniqueName: \"kubernetes.io/projected/a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8-kube-api-access-zrk5t\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:13:00.379667 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:00.379629    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52w9l" event={"ID":"a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8","Type":"ContainerDied","Data":"c89c68995df1c10da5d7b0015075169e92727cf33d59b511010be5cd2de5f38e"}
Jul 07 17:13:00.379667 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:00.379667    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af52w9l"
Jul 07 17:13:00.379846 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:00.379712    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c89c68995df1c10da5d7b0015075169e92727cf33d59b511010be5cd2de5f38e"
Jul 07 17:13:10.538745 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:10.538712    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-547959ffc4-rs7zv"]
Jul 07 17:13:10.539131 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:10.539010    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8" containerName="extract"
Jul 07 17:13:10.539131 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:10.539021    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8" containerName="extract"
Jul 07 17:13:10.539131 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:10.539031    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8" containerName="pull"
Jul 07 17:13:10.539131 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:10.539036    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8" containerName="pull"
Jul 07 17:13:10.539131 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:10.539057    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8" containerName="util"
Jul 07 17:13:10.539131 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:10.539062    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8" containerName="util"
Jul 07 17:13:10.539131 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:10.539104    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3ccf3ee-d3e0-4848-802c-f1cd4d0501a8" containerName="extract"
Jul 07 17:13:10.541261 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:10.541246    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-547959ffc4-rs7zv"
Jul 07 17:13:10.547012 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:10.546983    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 07 17:13:10.547631 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:10.547350    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 07 17:13:10.547710 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:10.547637    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 07 17:13:10.547931 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:10.547913    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 07 17:13:10.547931 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:10.547926    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-kchrk\""
Jul 07 17:13:10.562770 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:10.562745    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-547959ffc4-rs7zv"]
Jul 07 17:13:10.687133 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:10.687104    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6995de0d-adeb-44ea-8b4c-52d7464731bd-webhook-cert\") pod \"opendatahub-operator-controller-manager-547959ffc4-rs7zv\" (UID: \"6995de0d-adeb-44ea-8b4c-52d7464731bd\") " pod="opendatahub/opendatahub-operator-controller-manager-547959ffc4-rs7zv"
Jul 07 17:13:10.687350 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:10.687138    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4lhw\" (UniqueName: \"kubernetes.io/projected/6995de0d-adeb-44ea-8b4c-52d7464731bd-kube-api-access-h4lhw\") pod \"opendatahub-operator-controller-manager-547959ffc4-rs7zv\" (UID: \"6995de0d-adeb-44ea-8b4c-52d7464731bd\") " pod="opendatahub/opendatahub-operator-controller-manager-547959ffc4-rs7zv"
Jul 07 17:13:10.687350 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:10.687158    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6995de0d-adeb-44ea-8b4c-52d7464731bd-manifests\") pod \"opendatahub-operator-controller-manager-547959ffc4-rs7zv\" (UID: \"6995de0d-adeb-44ea-8b4c-52d7464731bd\") " pod="opendatahub/opendatahub-operator-controller-manager-547959ffc4-rs7zv"
Jul 07 17:13:10.687350 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:10.687242    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6995de0d-adeb-44ea-8b4c-52d7464731bd-tmp\") pod \"opendatahub-operator-controller-manager-547959ffc4-rs7zv\" (UID: \"6995de0d-adeb-44ea-8b4c-52d7464731bd\") " pod="opendatahub/opendatahub-operator-controller-manager-547959ffc4-rs7zv"
Jul 07 17:13:10.687479 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:10.687351    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6995de0d-adeb-44ea-8b4c-52d7464731bd-apiservice-cert\") pod \"opendatahub-operator-controller-manager-547959ffc4-rs7zv\" (UID: \"6995de0d-adeb-44ea-8b4c-52d7464731bd\") " pod="opendatahub/opendatahub-operator-controller-manager-547959ffc4-rs7zv"
Jul 07 17:13:10.788201 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:10.788172    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6995de0d-adeb-44ea-8b4c-52d7464731bd-apiservice-cert\") pod \"opendatahub-operator-controller-manager-547959ffc4-rs7zv\" (UID: \"6995de0d-adeb-44ea-8b4c-52d7464731bd\") " pod="opendatahub/opendatahub-operator-controller-manager-547959ffc4-rs7zv"
Jul 07 17:13:10.788322 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:10.788205    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6995de0d-adeb-44ea-8b4c-52d7464731bd-webhook-cert\") pod \"opendatahub-operator-controller-manager-547959ffc4-rs7zv\" (UID: \"6995de0d-adeb-44ea-8b4c-52d7464731bd\") " pod="opendatahub/opendatahub-operator-controller-manager-547959ffc4-rs7zv"
Jul 07 17:13:10.788322 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:10.788232    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4lhw\" (UniqueName: \"kubernetes.io/projected/6995de0d-adeb-44ea-8b4c-52d7464731bd-kube-api-access-h4lhw\") pod \"opendatahub-operator-controller-manager-547959ffc4-rs7zv\" (UID: \"6995de0d-adeb-44ea-8b4c-52d7464731bd\") " pod="opendatahub/opendatahub-operator-controller-manager-547959ffc4-rs7zv"
Jul 07 17:13:10.788322 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:10.788256    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6995de0d-adeb-44ea-8b4c-52d7464731bd-manifests\") pod \"opendatahub-operator-controller-manager-547959ffc4-rs7zv\" (UID: \"6995de0d-adeb-44ea-8b4c-52d7464731bd\") " pod="opendatahub/opendatahub-operator-controller-manager-547959ffc4-rs7zv"
Jul 07 17:13:10.788322 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:10.788301    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6995de0d-adeb-44ea-8b4c-52d7464731bd-tmp\") pod \"opendatahub-operator-controller-manager-547959ffc4-rs7zv\" (UID: \"6995de0d-adeb-44ea-8b4c-52d7464731bd\") " pod="opendatahub/opendatahub-operator-controller-manager-547959ffc4-rs7zv"
Jul 07 17:13:10.788814 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:10.788755    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6995de0d-adeb-44ea-8b4c-52d7464731bd-manifests\") pod \"opendatahub-operator-controller-manager-547959ffc4-rs7zv\" (UID: \"6995de0d-adeb-44ea-8b4c-52d7464731bd\") " pod="opendatahub/opendatahub-operator-controller-manager-547959ffc4-rs7zv"
Jul 07 17:13:10.790560 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:10.790541    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6995de0d-adeb-44ea-8b4c-52d7464731bd-tmp\") pod \"opendatahub-operator-controller-manager-547959ffc4-rs7zv\" (UID: \"6995de0d-adeb-44ea-8b4c-52d7464731bd\") " pod="opendatahub/opendatahub-operator-controller-manager-547959ffc4-rs7zv"
Jul 07 17:13:10.790753 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:10.790736    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6995de0d-adeb-44ea-8b4c-52d7464731bd-webhook-cert\") pod \"opendatahub-operator-controller-manager-547959ffc4-rs7zv\" (UID: \"6995de0d-adeb-44ea-8b4c-52d7464731bd\") " pod="opendatahub/opendatahub-operator-controller-manager-547959ffc4-rs7zv"
Jul 07 17:13:10.790797 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:10.790760    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6995de0d-adeb-44ea-8b4c-52d7464731bd-apiservice-cert\") pod \"opendatahub-operator-controller-manager-547959ffc4-rs7zv\" (UID: \"6995de0d-adeb-44ea-8b4c-52d7464731bd\") " pod="opendatahub/opendatahub-operator-controller-manager-547959ffc4-rs7zv"
Jul 07 17:13:10.797630 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:10.797610    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4lhw\" (UniqueName: \"kubernetes.io/projected/6995de0d-adeb-44ea-8b4c-52d7464731bd-kube-api-access-h4lhw\") pod \"opendatahub-operator-controller-manager-547959ffc4-rs7zv\" (UID: \"6995de0d-adeb-44ea-8b4c-52d7464731bd\") " pod="opendatahub/opendatahub-operator-controller-manager-547959ffc4-rs7zv"
Jul 07 17:13:10.851167 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:10.851146    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-547959ffc4-rs7zv"
Jul 07 17:13:10.985545 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:10.985504    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-547959ffc4-rs7zv"]
Jul 07 17:13:10.990010 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:13:10.989984    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6995de0d_adeb_44ea_8b4c_52d7464731bd.slice/crio-bbc38d3cb199be077777f74fce4738cf7cdcbbe2fd4184c4d7f0cd2ef51d5379 WatchSource:0}: Error finding container bbc38d3cb199be077777f74fce4738cf7cdcbbe2fd4184c4d7f0cd2ef51d5379: Status 404 returned error can't find the container with id bbc38d3cb199be077777f74fce4738cf7cdcbbe2fd4184c4d7f0cd2ef51d5379
Jul 07 17:13:10.991786 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:10.991769    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 17:13:11.425480 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:11.425443    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-547959ffc4-rs7zv" event={"ID":"6995de0d-adeb-44ea-8b4c-52d7464731bd","Type":"ContainerStarted","Data":"bbc38d3cb199be077777f74fce4738cf7cdcbbe2fd4184c4d7f0cd2ef51d5379"}
Jul 07 17:13:15.445427 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:15.445394    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-547959ffc4-rs7zv" event={"ID":"6995de0d-adeb-44ea-8b4c-52d7464731bd","Type":"ContainerStarted","Data":"068d65236dd4abb2404ac46bd39e4b91c1dc8f8d77688f93216f65da6637ccf7"}
Jul 07 17:13:16.451125 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:16.451092    2579 generic.go:356] "Generic (PLEG): container finished" podID="6995de0d-adeb-44ea-8b4c-52d7464731bd" containerID="068d65236dd4abb2404ac46bd39e4b91c1dc8f8d77688f93216f65da6637ccf7" exitCode=0
Jul 07 17:13:16.451504 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:16.451159    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-547959ffc4-rs7zv" event={"ID":"6995de0d-adeb-44ea-8b4c-52d7464731bd","Type":"ContainerDied","Data":"068d65236dd4abb2404ac46bd39e4b91c1dc8f8d77688f93216f65da6637ccf7"}
Jul 07 17:13:17.457853 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:17.457816    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-547959ffc4-rs7zv" event={"ID":"6995de0d-adeb-44ea-8b4c-52d7464731bd","Type":"ContainerStarted","Data":"a747603271ced7992bbebbfe8f4965d5686656f061d34fbafb75cca5786061b4"}
Jul 07 17:13:17.458214 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:17.457941    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-547959ffc4-rs7zv"
Jul 07 17:13:17.485041 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:17.484999    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-547959ffc4-rs7zv" podStartSLOduration=2.242783141 podStartE2EDuration="7.484987341s" totalImagesPullingTime="4.190289632s" totalInitContainerRuntime="1.051914568s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:13:10 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:13:17.484102368 +0000 UTC m=+320.980180876" watchObservedRunningTime="2026-07-07 17:13:17.484987341 +0000 UTC m=+320.981065849"
Jul 07 17:13:28.464680 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:28.464647    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-547959ffc4-rs7zv"
Jul 07 17:13:31.042340 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:31.042310    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-76676f7fb7-ghktc"]
Jul 07 17:13:31.050600 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:31.050577    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-76676f7fb7-ghktc"
Jul 07 17:13:31.053513 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:31.053492    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 07 17:13:31.054276 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:31.054255    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 07 17:13:31.054276 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:31.054269    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 07 17:13:31.054410 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:31.054257    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 07 17:13:31.056538 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:31.056517    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-sh26c\""
Jul 07 17:13:31.073554 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:31.073530    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-76676f7fb7-ghktc"]
Jul 07 17:13:31.234936 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:31.234914    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c201225c-e095-4494-b013-694764cd0272-tmp\") pod \"kube-auth-proxy-76676f7fb7-ghktc\" (UID: \"c201225c-e095-4494-b013-694764cd0272\") " pod="openshift-ingress/kube-auth-proxy-76676f7fb7-ghktc"
Jul 07 17:13:31.235067 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:31.234946    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5j7w\" (UniqueName: \"kubernetes.io/projected/c201225c-e095-4494-b013-694764cd0272-kube-api-access-k5j7w\") pod \"kube-auth-proxy-76676f7fb7-ghktc\" (UID: \"c201225c-e095-4494-b013-694764cd0272\") " pod="openshift-ingress/kube-auth-proxy-76676f7fb7-ghktc"
Jul 07 17:13:31.235067 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:31.235012    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c201225c-e095-4494-b013-694764cd0272-tls-certs\") pod \"kube-auth-proxy-76676f7fb7-ghktc\" (UID: \"c201225c-e095-4494-b013-694764cd0272\") " pod="openshift-ingress/kube-auth-proxy-76676f7fb7-ghktc"
Jul 07 17:13:31.336133 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:31.336079    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c201225c-e095-4494-b013-694764cd0272-tls-certs\") pod \"kube-auth-proxy-76676f7fb7-ghktc\" (UID: \"c201225c-e095-4494-b013-694764cd0272\") " pod="openshift-ingress/kube-auth-proxy-76676f7fb7-ghktc"
Jul 07 17:13:31.336133 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:31.336120    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c201225c-e095-4494-b013-694764cd0272-tmp\") pod \"kube-auth-proxy-76676f7fb7-ghktc\" (UID: \"c201225c-e095-4494-b013-694764cd0272\") " pod="openshift-ingress/kube-auth-proxy-76676f7fb7-ghktc"
Jul 07 17:13:31.336257 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:31.336139    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5j7w\" (UniqueName: \"kubernetes.io/projected/c201225c-e095-4494-b013-694764cd0272-kube-api-access-k5j7w\") pod \"kube-auth-proxy-76676f7fb7-ghktc\" (UID: \"c201225c-e095-4494-b013-694764cd0272\") " pod="openshift-ingress/kube-auth-proxy-76676f7fb7-ghktc"
Jul 07 17:13:31.338387 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:31.338363    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c201225c-e095-4494-b013-694764cd0272-tmp\") pod \"kube-auth-proxy-76676f7fb7-ghktc\" (UID: \"c201225c-e095-4494-b013-694764cd0272\") " pod="openshift-ingress/kube-auth-proxy-76676f7fb7-ghktc"
Jul 07 17:13:31.338556 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:31.338540    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c201225c-e095-4494-b013-694764cd0272-tls-certs\") pod \"kube-auth-proxy-76676f7fb7-ghktc\" (UID: \"c201225c-e095-4494-b013-694764cd0272\") " pod="openshift-ingress/kube-auth-proxy-76676f7fb7-ghktc"
Jul 07 17:13:31.350131 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:31.350106    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5j7w\" (UniqueName: \"kubernetes.io/projected/c201225c-e095-4494-b013-694764cd0272-kube-api-access-k5j7w\") pod \"kube-auth-proxy-76676f7fb7-ghktc\" (UID: \"c201225c-e095-4494-b013-694764cd0272\") " pod="openshift-ingress/kube-auth-proxy-76676f7fb7-ghktc"
Jul 07 17:13:31.359956 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:31.359930    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-76676f7fb7-ghktc"
Jul 07 17:13:31.498661 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:31.498628    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-76676f7fb7-ghktc"]
Jul 07 17:13:31.510762 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:13:31.510733    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc201225c_e095_4494_b013_694764cd0272.slice/crio-a374391291b6c0b314615d34e186d29a27625c0e308a3f2d93b0b54ef1269920 WatchSource:0}: Error finding container a374391291b6c0b314615d34e186d29a27625c0e308a3f2d93b0b54ef1269920: Status 404 returned error can't find the container with id a374391291b6c0b314615d34e186d29a27625c0e308a3f2d93b0b54ef1269920
Jul 07 17:13:31.726938 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:31.726912    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-clcxv"]
Jul 07 17:13:31.729938 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:31.729924    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-clcxv"
Jul 07 17:13:31.732364 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:31.732346    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-97l5h\""
Jul 07 17:13:31.732587 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:31.732573    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 07 17:13:31.749662 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:31.749642    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-clcxv"]
Jul 07 17:13:31.840610 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:31.840581    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrm8m\" (UniqueName: \"kubernetes.io/projected/cedceb3f-624f-4148-b0cb-529b9d1888d6-kube-api-access-wrm8m\") pod \"odh-model-controller-7998c67dc4-clcxv\" (UID: \"cedceb3f-624f-4148-b0cb-529b9d1888d6\") " pod="opendatahub/odh-model-controller-7998c67dc4-clcxv"
Jul 07 17:13:31.840747 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:31.840622    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cedceb3f-624f-4148-b0cb-529b9d1888d6-cert\") pod \"odh-model-controller-7998c67dc4-clcxv\" (UID: \"cedceb3f-624f-4148-b0cb-529b9d1888d6\") " pod="opendatahub/odh-model-controller-7998c67dc4-clcxv"
Jul 07 17:13:31.941986 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:31.941960    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrm8m\" (UniqueName: \"kubernetes.io/projected/cedceb3f-624f-4148-b0cb-529b9d1888d6-kube-api-access-wrm8m\") pod \"odh-model-controller-7998c67dc4-clcxv\" (UID: \"cedceb3f-624f-4148-b0cb-529b9d1888d6\") " pod="opendatahub/odh-model-controller-7998c67dc4-clcxv"
Jul 07 17:13:31.942118 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:31.941999    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cedceb3f-624f-4148-b0cb-529b9d1888d6-cert\") pod \"odh-model-controller-7998c67dc4-clcxv\" (UID: \"cedceb3f-624f-4148-b0cb-529b9d1888d6\") " pod="opendatahub/odh-model-controller-7998c67dc4-clcxv"
Jul 07 17:13:31.944486 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:31.944466    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cedceb3f-624f-4148-b0cb-529b9d1888d6-cert\") pod \"odh-model-controller-7998c67dc4-clcxv\" (UID: \"cedceb3f-624f-4148-b0cb-529b9d1888d6\") " pod="opendatahub/odh-model-controller-7998c67dc4-clcxv"
Jul 07 17:13:31.954699 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:31.954672    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrm8m\" (UniqueName: \"kubernetes.io/projected/cedceb3f-624f-4148-b0cb-529b9d1888d6-kube-api-access-wrm8m\") pod \"odh-model-controller-7998c67dc4-clcxv\" (UID: \"cedceb3f-624f-4148-b0cb-529b9d1888d6\") " pod="opendatahub/odh-model-controller-7998c67dc4-clcxv"
Jul 07 17:13:32.039651 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:32.039589    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-clcxv"
Jul 07 17:13:32.204707 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:32.204676    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-clcxv"]
Jul 07 17:13:32.209164 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:13:32.209143    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcedceb3f_624f_4148_b0cb_529b9d1888d6.slice/crio-5414286ced1840be40d145a8a5e1ae5a22e29ca3d9890ffbb308f0bd6337ce29 WatchSource:0}: Error finding container 5414286ced1840be40d145a8a5e1ae5a22e29ca3d9890ffbb308f0bd6337ce29: Status 404 returned error can't find the container with id 5414286ced1840be40d145a8a5e1ae5a22e29ca3d9890ffbb308f0bd6337ce29
Jul 07 17:13:32.519907 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:32.519849    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-76676f7fb7-ghktc" event={"ID":"c201225c-e095-4494-b013-694764cd0272","Type":"ContainerStarted","Data":"a374391291b6c0b314615d34e186d29a27625c0e308a3f2d93b0b54ef1269920"}
Jul 07 17:13:32.520880 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:32.520858    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-clcxv" event={"ID":"cedceb3f-624f-4148-b0cb-529b9d1888d6","Type":"ContainerStarted","Data":"5414286ced1840be40d145a8a5e1ae5a22e29ca3d9890ffbb308f0bd6337ce29"}
Jul 07 17:13:33.315457 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:33.315408    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-4nbnm"]
Jul 07 17:13:33.318092 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:33.318065    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-4nbnm"
Jul 07 17:13:33.322523 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:33.322499    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 07 17:13:33.339544 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:33.339522    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-4nbnm"]
Jul 07 17:13:33.457465 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:33.457429    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20330f0d-cf1b-489e-a8c7-866bfbd84e4f-tls-certs\") pod \"model-serving-api-54d89bd5fc-4nbnm\" (UID: \"20330f0d-cf1b-489e-a8c7-866bfbd84e4f\") " pod="opendatahub/model-serving-api-54d89bd5fc-4nbnm"
Jul 07 17:13:33.457630 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:33.457507    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njmms\" (UniqueName: \"kubernetes.io/projected/20330f0d-cf1b-489e-a8c7-866bfbd84e4f-kube-api-access-njmms\") pod \"model-serving-api-54d89bd5fc-4nbnm\" (UID: \"20330f0d-cf1b-489e-a8c7-866bfbd84e4f\") " pod="opendatahub/model-serving-api-54d89bd5fc-4nbnm"
Jul 07 17:13:33.558441 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:33.558411    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njmms\" (UniqueName: \"kubernetes.io/projected/20330f0d-cf1b-489e-a8c7-866bfbd84e4f-kube-api-access-njmms\") pod \"model-serving-api-54d89bd5fc-4nbnm\" (UID: \"20330f0d-cf1b-489e-a8c7-866bfbd84e4f\") " pod="opendatahub/model-serving-api-54d89bd5fc-4nbnm"
Jul 07 17:13:33.558601 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:33.558501    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20330f0d-cf1b-489e-a8c7-866bfbd84e4f-tls-certs\") pod \"model-serving-api-54d89bd5fc-4nbnm\" (UID: \"20330f0d-cf1b-489e-a8c7-866bfbd84e4f\") " pod="opendatahub/model-serving-api-54d89bd5fc-4nbnm"
Jul 07 17:13:33.558659 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:13:33.558624    2579 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 07 17:13:33.558700 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:13:33.558690    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20330f0d-cf1b-489e-a8c7-866bfbd84e4f-tls-certs podName:20330f0d-cf1b-489e-a8c7-866bfbd84e4f nodeName:}" failed. No retries permitted until 2026-07-07 17:13:34.058670478 +0000 UTC m=+337.554748968 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/20330f0d-cf1b-489e-a8c7-866bfbd84e4f-tls-certs") pod "model-serving-api-54d89bd5fc-4nbnm" (UID: "20330f0d-cf1b-489e-a8c7-866bfbd84e4f") : secret "model-serving-api-tls" not found
Jul 07 17:13:33.615456 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:33.615378    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-njmms\" (UniqueName: \"kubernetes.io/projected/20330f0d-cf1b-489e-a8c7-866bfbd84e4f-kube-api-access-njmms\") pod \"model-serving-api-54d89bd5fc-4nbnm\" (UID: \"20330f0d-cf1b-489e-a8c7-866bfbd84e4f\") " pod="opendatahub/model-serving-api-54d89bd5fc-4nbnm"
Jul 07 17:13:34.062705 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:34.062669    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20330f0d-cf1b-489e-a8c7-866bfbd84e4f-tls-certs\") pod \"model-serving-api-54d89bd5fc-4nbnm\" (UID: \"20330f0d-cf1b-489e-a8c7-866bfbd84e4f\") " pod="opendatahub/model-serving-api-54d89bd5fc-4nbnm"
Jul 07 17:13:34.065991 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:34.065961    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20330f0d-cf1b-489e-a8c7-866bfbd84e4f-tls-certs\") pod \"model-serving-api-54d89bd5fc-4nbnm\" (UID: \"20330f0d-cf1b-489e-a8c7-866bfbd84e4f\") " pod="opendatahub/model-serving-api-54d89bd5fc-4nbnm"
Jul 07 17:13:34.231873 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:34.231826    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-4nbnm"
Jul 07 17:13:36.208214 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:36.208184    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-4nbnm"]
Jul 07 17:13:36.211116 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:13:36.211087    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20330f0d_cf1b_489e_a8c7_866bfbd84e4f.slice/crio-c8a20aa772913a4f9384550f9eeed4822f10f84d56e1a156c09d4a66fa86c421 WatchSource:0}: Error finding container c8a20aa772913a4f9384550f9eeed4822f10f84d56e1a156c09d4a66fa86c421: Status 404 returned error can't find the container with id c8a20aa772913a4f9384550f9eeed4822f10f84d56e1a156c09d4a66fa86c421
Jul 07 17:13:36.541029 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:36.540997    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-4nbnm" event={"ID":"20330f0d-cf1b-489e-a8c7-866bfbd84e4f","Type":"ContainerStarted","Data":"c8a20aa772913a4f9384550f9eeed4822f10f84d56e1a156c09d4a66fa86c421"}
Jul 07 17:13:36.542516 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:36.542482    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-clcxv" event={"ID":"cedceb3f-624f-4148-b0cb-529b9d1888d6","Type":"ContainerStarted","Data":"49b1e6fc45b262caaf41f573d1ddf65eb0414b180c631d89d96336232e9d9ad4"}
Jul 07 17:13:36.542663 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:36.542640    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-clcxv"
Jul 07 17:13:36.544190 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:36.544164    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-76676f7fb7-ghktc" event={"ID":"c201225c-e095-4494-b013-694764cd0272","Type":"ContainerStarted","Data":"2e140149def73fd59078a98e03c7d4cec4cced1f5ea2e996ec112e3cc700509a"}
Jul 07 17:13:36.565090 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:36.565053    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-clcxv" podStartSLOduration=1.699444862 podStartE2EDuration="5.565038826s" totalImagesPullingTime="3.865593964s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:13:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:13:36.564416292 +0000 UTC m=+340.060494801" watchObservedRunningTime="2026-07-07 17:13:36.565038826 +0000 UTC m=+340.061117378"
Jul 07 17:13:36.585068 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:36.585032    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-76676f7fb7-ghktc" podStartSLOduration=2.025907059 podStartE2EDuration="6.585019247s" totalImagesPullingTime="4.559112188s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:13:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:13:36.582771611 +0000 UTC m=+340.078850118" watchObservedRunningTime="2026-07-07 17:13:36.585019247 +0000 UTC m=+340.081097756"
Jul 07 17:13:38.555044 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:38.555009    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-4nbnm" event={"ID":"20330f0d-cf1b-489e-a8c7-866bfbd84e4f","Type":"ContainerStarted","Data":"a010c96b0865361853af110ed6a01969248a2207aa5dfa58085152c832481f6b"}
Jul 07 17:13:38.555396 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:38.555133    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-4nbnm"
Jul 07 17:13:39.746226 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:39.746177    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-4nbnm" podStartSLOduration=4.865007081 podStartE2EDuration="6.746159914s" totalImagesPullingTime="1.881152833s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:13:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:13:38.617902842 +0000 UTC m=+342.113981342" watchObservedRunningTime="2026-07-07 17:13:39.746159914 +0000 UTC m=+343.242238422"
Jul 07 17:13:39.746588 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:39.746414    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5bbf97565b-hw8dw"]
Jul 07 17:13:39.749745 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:39.749730    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5bbf97565b-hw8dw"
Jul 07 17:13:39.754357 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:39.754336    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 07 17:13:39.755203 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:39.755145    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 07 17:13:39.755203 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:39.755171    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 07 17:13:39.755363 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:39.755174    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 07 17:13:39.755363 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:39.755233    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 07 17:13:39.755469 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:39.755457    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-7gz65\""
Jul 07 17:13:39.768766 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:39.768739    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5bbf97565b-hw8dw"]
Jul 07 17:13:39.811670 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:39.811645    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d3be45b7-0764-4b02-8ed1-7bae149537e5-cert\") pod \"lws-controller-manager-5bbf97565b-hw8dw\" (UID: \"d3be45b7-0764-4b02-8ed1-7bae149537e5\") " pod="openshift-lws-operator/lws-controller-manager-5bbf97565b-hw8dw"
Jul 07 17:13:39.811756 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:39.811690    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d3be45b7-0764-4b02-8ed1-7bae149537e5-manager-config\") pod \"lws-controller-manager-5bbf97565b-hw8dw\" (UID: \"d3be45b7-0764-4b02-8ed1-7bae149537e5\") " pod="openshift-lws-operator/lws-controller-manager-5bbf97565b-hw8dw"
Jul 07 17:13:39.811756 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:39.811748    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zhzq\" (UniqueName: \"kubernetes.io/projected/d3be45b7-0764-4b02-8ed1-7bae149537e5-kube-api-access-2zhzq\") pod \"lws-controller-manager-5bbf97565b-hw8dw\" (UID: \"d3be45b7-0764-4b02-8ed1-7bae149537e5\") " pod="openshift-lws-operator/lws-controller-manager-5bbf97565b-hw8dw"
Jul 07 17:13:39.811863 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:39.811791    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d3be45b7-0764-4b02-8ed1-7bae149537e5-metrics-cert\") pod \"lws-controller-manager-5bbf97565b-hw8dw\" (UID: \"d3be45b7-0764-4b02-8ed1-7bae149537e5\") " pod="openshift-lws-operator/lws-controller-manager-5bbf97565b-hw8dw"
Jul 07 17:13:39.913089 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:39.913068    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d3be45b7-0764-4b02-8ed1-7bae149537e5-cert\") pod \"lws-controller-manager-5bbf97565b-hw8dw\" (UID: \"d3be45b7-0764-4b02-8ed1-7bae149537e5\") " pod="openshift-lws-operator/lws-controller-manager-5bbf97565b-hw8dw"
Jul 07 17:13:39.913186 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:39.913108    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d3be45b7-0764-4b02-8ed1-7bae149537e5-manager-config\") pod \"lws-controller-manager-5bbf97565b-hw8dw\" (UID: \"d3be45b7-0764-4b02-8ed1-7bae149537e5\") " pod="openshift-lws-operator/lws-controller-manager-5bbf97565b-hw8dw"
Jul 07 17:13:39.913186 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:39.913131    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zhzq\" (UniqueName: \"kubernetes.io/projected/d3be45b7-0764-4b02-8ed1-7bae149537e5-kube-api-access-2zhzq\") pod \"lws-controller-manager-5bbf97565b-hw8dw\" (UID: \"d3be45b7-0764-4b02-8ed1-7bae149537e5\") " pod="openshift-lws-operator/lws-controller-manager-5bbf97565b-hw8dw"
Jul 07 17:13:39.913186 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:39.913153    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d3be45b7-0764-4b02-8ed1-7bae149537e5-metrics-cert\") pod \"lws-controller-manager-5bbf97565b-hw8dw\" (UID: \"d3be45b7-0764-4b02-8ed1-7bae149537e5\") " pod="openshift-lws-operator/lws-controller-manager-5bbf97565b-hw8dw"
Jul 07 17:13:39.913798 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:39.913775    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d3be45b7-0764-4b02-8ed1-7bae149537e5-manager-config\") pod \"lws-controller-manager-5bbf97565b-hw8dw\" (UID: \"d3be45b7-0764-4b02-8ed1-7bae149537e5\") " pod="openshift-lws-operator/lws-controller-manager-5bbf97565b-hw8dw"
Jul 07 17:13:39.915621 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:39.915594    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d3be45b7-0764-4b02-8ed1-7bae149537e5-cert\") pod \"lws-controller-manager-5bbf97565b-hw8dw\" (UID: \"d3be45b7-0764-4b02-8ed1-7bae149537e5\") " pod="openshift-lws-operator/lws-controller-manager-5bbf97565b-hw8dw"
Jul 07 17:13:39.915718 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:39.915636    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d3be45b7-0764-4b02-8ed1-7bae149537e5-metrics-cert\") pod \"lws-controller-manager-5bbf97565b-hw8dw\" (UID: \"d3be45b7-0764-4b02-8ed1-7bae149537e5\") " pod="openshift-lws-operator/lws-controller-manager-5bbf97565b-hw8dw"
Jul 07 17:13:39.924008 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:39.923989    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zhzq\" (UniqueName: \"kubernetes.io/projected/d3be45b7-0764-4b02-8ed1-7bae149537e5-kube-api-access-2zhzq\") pod \"lws-controller-manager-5bbf97565b-hw8dw\" (UID: \"d3be45b7-0764-4b02-8ed1-7bae149537e5\") " pod="openshift-lws-operator/lws-controller-manager-5bbf97565b-hw8dw"
Jul 07 17:13:40.058789 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:40.058728    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5bbf97565b-hw8dw"
Jul 07 17:13:40.243877 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:40.243853    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5bbf97565b-hw8dw"]
Jul 07 17:13:40.246291 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:13:40.246266    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3be45b7_0764_4b02_8ed1_7bae149537e5.slice/crio-2ec50e14f4d247d42afb19352bcc117051e098ddbba70957c0be624f67e37a60 WatchSource:0}: Error finding container 2ec50e14f4d247d42afb19352bcc117051e098ddbba70957c0be624f67e37a60: Status 404 returned error can't find the container with id 2ec50e14f4d247d42afb19352bcc117051e098ddbba70957c0be624f67e37a60
Jul 07 17:13:40.572631 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:40.572602    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5bbf97565b-hw8dw" event={"ID":"d3be45b7-0764-4b02-8ed1-7bae149537e5","Type":"ContainerStarted","Data":"2ec50e14f4d247d42afb19352bcc117051e098ddbba70957c0be624f67e37a60"}
Jul 07 17:13:42.584768 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:42.584735    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5bbf97565b-hw8dw" event={"ID":"d3be45b7-0764-4b02-8ed1-7bae149537e5","Type":"ContainerStarted","Data":"6ca07f28b653e584c8b2cd2932761f1050aed876e4c804cc0f4ff137589e948c"}
Jul 07 17:13:42.585176 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:42.584864    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5bbf97565b-hw8dw"
Jul 07 17:13:47.551103 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:47.551007    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-clcxv"
Jul 07 17:13:47.594987 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:47.594936    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5bbf97565b-hw8dw" podStartSLOduration=6.726300941 podStartE2EDuration="8.594916287s" totalImagesPullingTime="1.868615346s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:13:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:13:42.642681287 +0000 UTC m=+346.138759793" watchObservedRunningTime="2026-07-07 17:13:47.594916287 +0000 UTC m=+351.090994797"
Jul 07 17:13:49.565085 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:49.565053    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-4nbnm"
Jul 07 17:13:53.591672 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:13:53.591643    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5bbf97565b-hw8dw"
Jul 07 17:14:47.521410 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:47.521376    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04npwd"]
Jul 07 17:14:47.523568 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:47.523553    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04npwd"
Jul 07 17:14:47.526137 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:47.526105    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 07 17:14:47.526249 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:47.526105    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 07 17:14:47.526948 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:47.526933    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-vv4vg\""
Jul 07 17:14:47.534029 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:47.534007    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04npwd"]
Jul 07 17:14:47.694474 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:47.694444    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ff5c3733-6971-4414-9c3d-87d9676bea7c-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04npwd\" (UID: \"ff5c3733-6971-4414-9c3d-87d9676bea7c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04npwd"
Jul 07 17:14:47.694677 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:47.694483    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrx8s\" (UniqueName: \"kubernetes.io/projected/ff5c3733-6971-4414-9c3d-87d9676bea7c-kube-api-access-wrx8s\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04npwd\" (UID: \"ff5c3733-6971-4414-9c3d-87d9676bea7c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04npwd"
Jul 07 17:14:47.694677 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:47.694532    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ff5c3733-6971-4414-9c3d-87d9676bea7c-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04npwd\" (UID: \"ff5c3733-6971-4414-9c3d-87d9676bea7c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04npwd"
Jul 07 17:14:47.795158 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:47.795098    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrx8s\" (UniqueName: \"kubernetes.io/projected/ff5c3733-6971-4414-9c3d-87d9676bea7c-kube-api-access-wrx8s\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04npwd\" (UID: \"ff5c3733-6971-4414-9c3d-87d9676bea7c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04npwd"
Jul 07 17:14:47.795158 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:47.795132    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ff5c3733-6971-4414-9c3d-87d9676bea7c-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04npwd\" (UID: \"ff5c3733-6971-4414-9c3d-87d9676bea7c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04npwd"
Jul 07 17:14:47.795324 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:47.795216    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ff5c3733-6971-4414-9c3d-87d9676bea7c-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04npwd\" (UID: \"ff5c3733-6971-4414-9c3d-87d9676bea7c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04npwd"
Jul 07 17:14:47.795515 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:47.795487    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ff5c3733-6971-4414-9c3d-87d9676bea7c-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04npwd\" (UID: \"ff5c3733-6971-4414-9c3d-87d9676bea7c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04npwd"
Jul 07 17:14:47.795575 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:47.795523    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ff5c3733-6971-4414-9c3d-87d9676bea7c-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04npwd\" (UID: \"ff5c3733-6971-4414-9c3d-87d9676bea7c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04npwd"
Jul 07 17:14:47.805226 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:47.805206    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrx8s\" (UniqueName: \"kubernetes.io/projected/ff5c3733-6971-4414-9c3d-87d9676bea7c-kube-api-access-wrx8s\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04npwd\" (UID: \"ff5c3733-6971-4414-9c3d-87d9676bea7c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04npwd"
Jul 07 17:14:47.832717 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:47.832698    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04npwd"
Jul 07 17:14:47.923846 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:47.923816    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75985h85"]
Jul 07 17:14:47.926785 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:47.926758    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75985h85"
Jul 07 17:14:47.940341 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:47.940318    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75985h85"]
Jul 07 17:14:47.973022 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:47.973003    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04npwd"]
Jul 07 17:14:47.975084 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:14:47.975058    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff5c3733_6971_4414_9c3d_87d9676bea7c.slice/crio-37ee59966f837979a4a4a3abfff67b82b7109d86adbba6be646492a544e9fd54 WatchSource:0}: Error finding container 37ee59966f837979a4a4a3abfff67b82b7109d86adbba6be646492a544e9fd54: Status 404 returned error can't find the container with id 37ee59966f837979a4a4a3abfff67b82b7109d86adbba6be646492a544e9fd54
Jul 07 17:14:48.098170 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:48.098140    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2bc00da5-eced-45fb-ac31-62874e629de7-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75985h85\" (UID: \"2bc00da5-eced-45fb-ac31-62874e629de7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75985h85"
Jul 07 17:14:48.098293 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:48.098213    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fgj5\" (UniqueName: \"kubernetes.io/projected/2bc00da5-eced-45fb-ac31-62874e629de7-kube-api-access-7fgj5\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75985h85\" (UID: \"2bc00da5-eced-45fb-ac31-62874e629de7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75985h85"
Jul 07 17:14:48.098293 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:48.098247    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2bc00da5-eced-45fb-ac31-62874e629de7-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75985h85\" (UID: \"2bc00da5-eced-45fb-ac31-62874e629de7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75985h85"
Jul 07 17:14:48.198870 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:48.198844    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fgj5\" (UniqueName: \"kubernetes.io/projected/2bc00da5-eced-45fb-ac31-62874e629de7-kube-api-access-7fgj5\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75985h85\" (UID: \"2bc00da5-eced-45fb-ac31-62874e629de7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75985h85"
Jul 07 17:14:48.198997 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:48.198876    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2bc00da5-eced-45fb-ac31-62874e629de7-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75985h85\" (UID: \"2bc00da5-eced-45fb-ac31-62874e629de7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75985h85"
Jul 07 17:14:48.198997 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:48.198938    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2bc00da5-eced-45fb-ac31-62874e629de7-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75985h85\" (UID: \"2bc00da5-eced-45fb-ac31-62874e629de7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75985h85"
Jul 07 17:14:48.199209 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:48.199195    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2bc00da5-eced-45fb-ac31-62874e629de7-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75985h85\" (UID: \"2bc00da5-eced-45fb-ac31-62874e629de7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75985h85"
Jul 07 17:14:48.199261 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:48.199235    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2bc00da5-eced-45fb-ac31-62874e629de7-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75985h85\" (UID: \"2bc00da5-eced-45fb-ac31-62874e629de7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75985h85"
Jul 07 17:14:48.208509 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:48.208491    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fgj5\" (UniqueName: \"kubernetes.io/projected/2bc00da5-eced-45fb-ac31-62874e629de7-kube-api-access-7fgj5\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75985h85\" (UID: \"2bc00da5-eced-45fb-ac31-62874e629de7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75985h85"
Jul 07 17:14:48.239987 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:48.239969    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75985h85"
Jul 07 17:14:48.324421 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:48.324391    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvk57"]
Jul 07 17:14:48.326926 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:48.326906    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvk57"
Jul 07 17:14:48.338291 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:48.338268    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvk57"]
Jul 07 17:14:48.370765 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:48.370746    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75985h85"]
Jul 07 17:14:48.372984 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:14:48.372957    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bc00da5_eced_45fb_ac31_62874e629de7.slice/crio-bf9e25275b9b186b711b81469339ebaf65c11155e7f483096a8d63f42fb0636a WatchSource:0}: Error finding container bf9e25275b9b186b711b81469339ebaf65c11155e7f483096a8d63f42fb0636a: Status 404 returned error can't find the container with id bf9e25275b9b186b711b81469339ebaf65c11155e7f483096a8d63f42fb0636a
Jul 07 17:14:48.400854 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:48.400833    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8tsm\" (UniqueName: \"kubernetes.io/projected/f0112e35-a82a-43c2-9d3f-02613ca5e3e4-kube-api-access-s8tsm\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvk57\" (UID: \"f0112e35-a82a-43c2-9d3f-02613ca5e3e4\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvk57"
Jul 07 17:14:48.400957 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:48.400878    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f0112e35-a82a-43c2-9d3f-02613ca5e3e4-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvk57\" (UID: \"f0112e35-a82a-43c2-9d3f-02613ca5e3e4\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvk57"
Jul 07 17:14:48.400999 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:48.400958    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f0112e35-a82a-43c2-9d3f-02613ca5e3e4-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvk57\" (UID: \"f0112e35-a82a-43c2-9d3f-02613ca5e3e4\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvk57"
Jul 07 17:14:48.501484 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:48.501456    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f0112e35-a82a-43c2-9d3f-02613ca5e3e4-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvk57\" (UID: \"f0112e35-a82a-43c2-9d3f-02613ca5e3e4\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvk57"
Jul 07 17:14:48.501584 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:48.501503    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f0112e35-a82a-43c2-9d3f-02613ca5e3e4-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvk57\" (UID: \"f0112e35-a82a-43c2-9d3f-02613ca5e3e4\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvk57"
Jul 07 17:14:48.501584 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:48.501537    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8tsm\" (UniqueName: \"kubernetes.io/projected/f0112e35-a82a-43c2-9d3f-02613ca5e3e4-kube-api-access-s8tsm\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvk57\" (UID: \"f0112e35-a82a-43c2-9d3f-02613ca5e3e4\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvk57"
Jul 07 17:14:48.501857 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:48.501836    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f0112e35-a82a-43c2-9d3f-02613ca5e3e4-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvk57\" (UID: \"f0112e35-a82a-43c2-9d3f-02613ca5e3e4\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvk57"
Jul 07 17:14:48.501917 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:48.501844    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f0112e35-a82a-43c2-9d3f-02613ca5e3e4-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvk57\" (UID: \"f0112e35-a82a-43c2-9d3f-02613ca5e3e4\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvk57"
Jul 07 17:14:48.511250 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:48.511225    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8tsm\" (UniqueName: \"kubernetes.io/projected/f0112e35-a82a-43c2-9d3f-02613ca5e3e4-kube-api-access-s8tsm\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvk57\" (UID: \"f0112e35-a82a-43c2-9d3f-02613ca5e3e4\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvk57"
Jul 07 17:14:48.637394 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:48.637337    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvk57"
Jul 07 17:14:48.729030 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:48.729003    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef199k25"]
Jul 07 17:14:48.732096 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:48.732069    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef199k25"
Jul 07 17:14:48.742623 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:48.742598    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef199k25"]
Jul 07 17:14:48.765382 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:48.765361    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvk57"]
Jul 07 17:14:48.767006 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:14:48.766985    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0112e35_a82a_43c2_9d3f_02613ca5e3e4.slice/crio-a1c5af041c6f62ad2f38ebdbaca7e2665ffb690e265c667c22b3d22e504b872a WatchSource:0}: Error finding container a1c5af041c6f62ad2f38ebdbaca7e2665ffb690e265c667c22b3d22e504b872a: Status 404 returned error can't find the container with id a1c5af041c6f62ad2f38ebdbaca7e2665ffb690e265c667c22b3d22e504b872a
Jul 07 17:14:48.880960 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:48.880924    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvk57" event={"ID":"f0112e35-a82a-43c2-9d3f-02613ca5e3e4","Type":"ContainerStarted","Data":"e7c25def5e20d53b04a61757369641a5c5ba2539ac9e591e07524994331dfe39"}
Jul 07 17:14:48.881124 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:48.880979    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvk57" event={"ID":"f0112e35-a82a-43c2-9d3f-02613ca5e3e4","Type":"ContainerStarted","Data":"a1c5af041c6f62ad2f38ebdbaca7e2665ffb690e265c667c22b3d22e504b872a"}
Jul 07 17:14:48.882419 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:48.882388    2579 generic.go:356] "Generic (PLEG): container finished" podID="ff5c3733-6971-4414-9c3d-87d9676bea7c" containerID="fff5863e54ea716e4730e8cd832a5cf82bc5de865a7cdbc750036b835f502e0f" exitCode=0
Jul 07 17:14:48.882504 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:48.882472    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04npwd" event={"ID":"ff5c3733-6971-4414-9c3d-87d9676bea7c","Type":"ContainerDied","Data":"fff5863e54ea716e4730e8cd832a5cf82bc5de865a7cdbc750036b835f502e0f"}
Jul 07 17:14:48.882552 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:48.882513    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04npwd" event={"ID":"ff5c3733-6971-4414-9c3d-87d9676bea7c","Type":"ContainerStarted","Data":"37ee59966f837979a4a4a3abfff67b82b7109d86adbba6be646492a544e9fd54"}
Jul 07 17:14:48.884151 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:48.884124    2579 generic.go:356] "Generic (PLEG): container finished" podID="2bc00da5-eced-45fb-ac31-62874e629de7" containerID="c6b847e22e61b6b9f77f3e8e5c5a8bf8ebb8bc5ea6cb5a7cbf856398ae68292b" exitCode=0
Jul 07 17:14:48.884262 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:48.884180    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75985h85" event={"ID":"2bc00da5-eced-45fb-ac31-62874e629de7","Type":"ContainerDied","Data":"c6b847e22e61b6b9f77f3e8e5c5a8bf8ebb8bc5ea6cb5a7cbf856398ae68292b"}
Jul 07 17:14:48.884262 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:48.884219    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75985h85" event={"ID":"2bc00da5-eced-45fb-ac31-62874e629de7","Type":"ContainerStarted","Data":"bf9e25275b9b186b711b81469339ebaf65c11155e7f483096a8d63f42fb0636a"}
Jul 07 17:14:48.905534 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:48.905509    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/218a3e4d-650c-4a0c-bd60-1670cffe7474-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef199k25\" (UID: \"218a3e4d-650c-4a0c-bd60-1670cffe7474\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef199k25"
Jul 07 17:14:48.905632 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:48.905545    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/218a3e4d-650c-4a0c-bd60-1670cffe7474-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef199k25\" (UID: \"218a3e4d-650c-4a0c-bd60-1670cffe7474\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef199k25"
Jul 07 17:14:48.905696 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:48.905633    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prrj5\" (UniqueName: \"kubernetes.io/projected/218a3e4d-650c-4a0c-bd60-1670cffe7474-kube-api-access-prrj5\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef199k25\" (UID: \"218a3e4d-650c-4a0c-bd60-1670cffe7474\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef199k25"
Jul 07 17:14:49.007026 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:49.007008    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/218a3e4d-650c-4a0c-bd60-1670cffe7474-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef199k25\" (UID: \"218a3e4d-650c-4a0c-bd60-1670cffe7474\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef199k25"
Jul 07 17:14:49.007137 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:49.007041    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/218a3e4d-650c-4a0c-bd60-1670cffe7474-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef199k25\" (UID: \"218a3e4d-650c-4a0c-bd60-1670cffe7474\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef199k25"
Jul 07 17:14:49.007137 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:49.007103    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prrj5\" (UniqueName: \"kubernetes.io/projected/218a3e4d-650c-4a0c-bd60-1670cffe7474-kube-api-access-prrj5\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef199k25\" (UID: \"218a3e4d-650c-4a0c-bd60-1670cffe7474\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef199k25"
Jul 07 17:14:49.007369 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:49.007351    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/218a3e4d-650c-4a0c-bd60-1670cffe7474-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef199k25\" (UID: \"218a3e4d-650c-4a0c-bd60-1670cffe7474\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef199k25"
Jul 07 17:14:49.007450 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:49.007398    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/218a3e4d-650c-4a0c-bd60-1670cffe7474-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef199k25\" (UID: \"218a3e4d-650c-4a0c-bd60-1670cffe7474\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef199k25"
Jul 07 17:14:49.016575 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:49.016553    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-prrj5\" (UniqueName: \"kubernetes.io/projected/218a3e4d-650c-4a0c-bd60-1670cffe7474-kube-api-access-prrj5\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef199k25\" (UID: \"218a3e4d-650c-4a0c-bd60-1670cffe7474\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef199k25"
Jul 07 17:14:49.042496 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:49.042479    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef199k25"
Jul 07 17:14:49.165110 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:49.165045    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef199k25"]
Jul 07 17:14:49.167600 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:14:49.167577    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod218a3e4d_650c_4a0c_bd60_1670cffe7474.slice/crio-361fe09e884d73cc2916f22e7aa678879c6e2a5df72d4545820cd023ec1a0a58 WatchSource:0}: Error finding container 361fe09e884d73cc2916f22e7aa678879c6e2a5df72d4545820cd023ec1a0a58: Status 404 returned error can't find the container with id 361fe09e884d73cc2916f22e7aa678879c6e2a5df72d4545820cd023ec1a0a58
Jul 07 17:14:49.890593 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:49.890565    2579 generic.go:356] "Generic (PLEG): container finished" podID="218a3e4d-650c-4a0c-bd60-1670cffe7474" containerID="e503babeaddac410bf31d32a7508df872eb1dae342a6e3ddb40d70a422b0220a" exitCode=0
Jul 07 17:14:49.890937 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:49.890643    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef199k25" event={"ID":"218a3e4d-650c-4a0c-bd60-1670cffe7474","Type":"ContainerDied","Data":"e503babeaddac410bf31d32a7508df872eb1dae342a6e3ddb40d70a422b0220a"}
Jul 07 17:14:49.890937 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:49.890679    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef199k25" event={"ID":"218a3e4d-650c-4a0c-bd60-1670cffe7474","Type":"ContainerStarted","Data":"361fe09e884d73cc2916f22e7aa678879c6e2a5df72d4545820cd023ec1a0a58"}
Jul 07 17:14:49.892123 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:49.892102    2579 generic.go:356] "Generic (PLEG): container finished" podID="f0112e35-a82a-43c2-9d3f-02613ca5e3e4" containerID="e7c25def5e20d53b04a61757369641a5c5ba2539ac9e591e07524994331dfe39" exitCode=0
Jul 07 17:14:49.892240 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:49.892141    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvk57" event={"ID":"f0112e35-a82a-43c2-9d3f-02613ca5e3e4","Type":"ContainerDied","Data":"e7c25def5e20d53b04a61757369641a5c5ba2539ac9e591e07524994331dfe39"}
Jul 07 17:14:49.894063 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:49.894026    2579 generic.go:356] "Generic (PLEG): container finished" podID="ff5c3733-6971-4414-9c3d-87d9676bea7c" containerID="f982a06ee5bdedefdfa49b6440f8d45c4fdc47d18470458a6ae24a286dffe907" exitCode=0
Jul 07 17:14:49.894190 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:49.894101    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04npwd" event={"ID":"ff5c3733-6971-4414-9c3d-87d9676bea7c","Type":"ContainerDied","Data":"f982a06ee5bdedefdfa49b6440f8d45c4fdc47d18470458a6ae24a286dffe907"}
Jul 07 17:14:49.896272 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:49.896255    2579 generic.go:356] "Generic (PLEG): container finished" podID="2bc00da5-eced-45fb-ac31-62874e629de7" containerID="0af51ab40125c906af4f3b41aed617929a9fb05ea1ac59ce632e33f14d8175a3" exitCode=0
Jul 07 17:14:49.896359 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:49.896283    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75985h85" event={"ID":"2bc00da5-eced-45fb-ac31-62874e629de7","Type":"ContainerDied","Data":"0af51ab40125c906af4f3b41aed617929a9fb05ea1ac59ce632e33f14d8175a3"}
Jul 07 17:14:50.903494 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:50.903459    2579 generic.go:356] "Generic (PLEG): container finished" podID="2bc00da5-eced-45fb-ac31-62874e629de7" containerID="b04df3599a36d654350b935d8a3c4795e3d971ce4b96992184d6813e44968fc5" exitCode=0
Jul 07 17:14:50.903840 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:50.903553    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75985h85" event={"ID":"2bc00da5-eced-45fb-ac31-62874e629de7","Type":"ContainerDied","Data":"b04df3599a36d654350b935d8a3c4795e3d971ce4b96992184d6813e44968fc5"}
Jul 07 17:14:50.905185 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:50.905164    2579 generic.go:356] "Generic (PLEG): container finished" podID="218a3e4d-650c-4a0c-bd60-1670cffe7474" containerID="94484fecf542a31d7584563da23506104885a1168fe68ee1b68c6e00f5e27f90" exitCode=0
Jul 07 17:14:50.905284 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:50.905244    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef199k25" event={"ID":"218a3e4d-650c-4a0c-bd60-1670cffe7474","Type":"ContainerDied","Data":"94484fecf542a31d7584563da23506104885a1168fe68ee1b68c6e00f5e27f90"}
Jul 07 17:14:50.907001 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:50.906983    2579 generic.go:356] "Generic (PLEG): container finished" podID="f0112e35-a82a-43c2-9d3f-02613ca5e3e4" containerID="d3ab0d4a030f5c7f2d58940656c79a7ec74622c7d64b4af27a41dcea0d103023" exitCode=0
Jul 07 17:14:50.907290 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:50.907052    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvk57" event={"ID":"f0112e35-a82a-43c2-9d3f-02613ca5e3e4","Type":"ContainerDied","Data":"d3ab0d4a030f5c7f2d58940656c79a7ec74622c7d64b4af27a41dcea0d103023"}
Jul 07 17:14:50.909237 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:50.909218    2579 generic.go:356] "Generic (PLEG): container finished" podID="ff5c3733-6971-4414-9c3d-87d9676bea7c" containerID="0a5908ee00f25ed8756f37bf1a1ef6bef5b571e2e5e6e4f6c74f350944e20a8a" exitCode=0
Jul 07 17:14:50.909325 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:50.909290    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04npwd" event={"ID":"ff5c3733-6971-4414-9c3d-87d9676bea7c","Type":"ContainerDied","Data":"0a5908ee00f25ed8756f37bf1a1ef6bef5b571e2e5e6e4f6c74f350944e20a8a"}
Jul 07 17:14:51.922081 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:51.922052    2579 generic.go:356] "Generic (PLEG): container finished" podID="f0112e35-a82a-43c2-9d3f-02613ca5e3e4" containerID="1b8004c34d07a0320b63da9e96d028f5ccbc4a4226351462c544c0477dc853ed" exitCode=0
Jul 07 17:14:51.922459 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:51.922190    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvk57" event={"ID":"f0112e35-a82a-43c2-9d3f-02613ca5e3e4","Type":"ContainerDied","Data":"1b8004c34d07a0320b63da9e96d028f5ccbc4a4226351462c544c0477dc853ed"}
Jul 07 17:14:51.924041 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:51.924017    2579 generic.go:356] "Generic (PLEG): container finished" podID="218a3e4d-650c-4a0c-bd60-1670cffe7474" containerID="a1e1b90af4c32cd72587039daa9f93f1956cb496d613731a5b1be01928fc5707" exitCode=0
Jul 07 17:14:51.924163 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:51.924086    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef199k25" event={"ID":"218a3e4d-650c-4a0c-bd60-1670cffe7474","Type":"ContainerDied","Data":"a1e1b90af4c32cd72587039daa9f93f1956cb496d613731a5b1be01928fc5707"}
Jul 07 17:14:52.073848 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:52.073826    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04npwd"
Jul 07 17:14:52.076369 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:52.076351    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75985h85"
Jul 07 17:14:52.231832 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:52.231766    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2bc00da5-eced-45fb-ac31-62874e629de7-bundle\") pod \"2bc00da5-eced-45fb-ac31-62874e629de7\" (UID: \"2bc00da5-eced-45fb-ac31-62874e629de7\") "
Jul 07 17:14:52.231832 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:52.231809    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7fgj5\" (UniqueName: \"kubernetes.io/projected/2bc00da5-eced-45fb-ac31-62874e629de7-kube-api-access-7fgj5\") pod \"2bc00da5-eced-45fb-ac31-62874e629de7\" (UID: \"2bc00da5-eced-45fb-ac31-62874e629de7\") "
Jul 07 17:14:52.231996 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:52.231839    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wrx8s\" (UniqueName: \"kubernetes.io/projected/ff5c3733-6971-4414-9c3d-87d9676bea7c-kube-api-access-wrx8s\") pod \"ff5c3733-6971-4414-9c3d-87d9676bea7c\" (UID: \"ff5c3733-6971-4414-9c3d-87d9676bea7c\") "
Jul 07 17:14:52.231996 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:52.231868    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ff5c3733-6971-4414-9c3d-87d9676bea7c-bundle\") pod \"ff5c3733-6971-4414-9c3d-87d9676bea7c\" (UID: \"ff5c3733-6971-4414-9c3d-87d9676bea7c\") "
Jul 07 17:14:52.231996 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:52.231915    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ff5c3733-6971-4414-9c3d-87d9676bea7c-util\") pod \"ff5c3733-6971-4414-9c3d-87d9676bea7c\" (UID: \"ff5c3733-6971-4414-9c3d-87d9676bea7c\") "
Jul 07 17:14:52.231996 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:52.231957    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2bc00da5-eced-45fb-ac31-62874e629de7-util\") pod \"2bc00da5-eced-45fb-ac31-62874e629de7\" (UID: \"2bc00da5-eced-45fb-ac31-62874e629de7\") "
Jul 07 17:14:52.232368 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:52.232330    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bc00da5-eced-45fb-ac31-62874e629de7-bundle" (OuterVolumeSpecName: "bundle") pod "2bc00da5-eced-45fb-ac31-62874e629de7" (UID: "2bc00da5-eced-45fb-ac31-62874e629de7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:14:52.232496 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:52.232402    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff5c3733-6971-4414-9c3d-87d9676bea7c-bundle" (OuterVolumeSpecName: "bundle") pod "ff5c3733-6971-4414-9c3d-87d9676bea7c" (UID: "ff5c3733-6971-4414-9c3d-87d9676bea7c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:14:52.234039 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:52.234018    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bc00da5-eced-45fb-ac31-62874e629de7-kube-api-access-7fgj5" (OuterVolumeSpecName: "kube-api-access-7fgj5") pod "2bc00da5-eced-45fb-ac31-62874e629de7" (UID: "2bc00da5-eced-45fb-ac31-62874e629de7"). InnerVolumeSpecName "kube-api-access-7fgj5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 17:14:52.234318 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:52.234304    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff5c3733-6971-4414-9c3d-87d9676bea7c-kube-api-access-wrx8s" (OuterVolumeSpecName: "kube-api-access-wrx8s") pod "ff5c3733-6971-4414-9c3d-87d9676bea7c" (UID: "ff5c3733-6971-4414-9c3d-87d9676bea7c"). InnerVolumeSpecName "kube-api-access-wrx8s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 17:14:52.237849 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:52.237827    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff5c3733-6971-4414-9c3d-87d9676bea7c-util" (OuterVolumeSpecName: "util") pod "ff5c3733-6971-4414-9c3d-87d9676bea7c" (UID: "ff5c3733-6971-4414-9c3d-87d9676bea7c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:14:52.238612 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:52.238564    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bc00da5-eced-45fb-ac31-62874e629de7-util" (OuterVolumeSpecName: "util") pod "2bc00da5-eced-45fb-ac31-62874e629de7" (UID: "2bc00da5-eced-45fb-ac31-62874e629de7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:14:52.333453 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:52.333431    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2bc00da5-eced-45fb-ac31-62874e629de7-bundle\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:14:52.333453 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:52.333452    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7fgj5\" (UniqueName: \"kubernetes.io/projected/2bc00da5-eced-45fb-ac31-62874e629de7-kube-api-access-7fgj5\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:14:52.333555 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:52.333463    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wrx8s\" (UniqueName: \"kubernetes.io/projected/ff5c3733-6971-4414-9c3d-87d9676bea7c-kube-api-access-wrx8s\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:14:52.333555 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:52.333472    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ff5c3733-6971-4414-9c3d-87d9676bea7c-bundle\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:14:52.333555 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:52.333480    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ff5c3733-6971-4414-9c3d-87d9676bea7c-util\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:14:52.333555 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:52.333487    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2bc00da5-eced-45fb-ac31-62874e629de7-util\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:14:52.930758 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:52.930735    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75985h85"
Jul 07 17:14:52.931231 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:52.930767    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75985h85" event={"ID":"2bc00da5-eced-45fb-ac31-62874e629de7","Type":"ContainerDied","Data":"bf9e25275b9b186b711b81469339ebaf65c11155e7f483096a8d63f42fb0636a"}
Jul 07 17:14:52.931231 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:52.930807    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bf9e25275b9b186b711b81469339ebaf65c11155e7f483096a8d63f42fb0636a"
Jul 07 17:14:52.932457 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:52.932422    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04npwd" event={"ID":"ff5c3733-6971-4414-9c3d-87d9676bea7c","Type":"ContainerDied","Data":"37ee59966f837979a4a4a3abfff67b82b7109d86adbba6be646492a544e9fd54"}
Jul 07 17:14:52.932565 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:52.932476    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="37ee59966f837979a4a4a3abfff67b82b7109d86adbba6be646492a544e9fd54"
Jul 07 17:14:52.932714 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:52.932699    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04npwd"
Jul 07 17:14:53.048509 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:53.048485    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef199k25"
Jul 07 17:14:53.086473 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:53.086453    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvk57"
Jul 07 17:14:53.139116 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:53.139095    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/218a3e4d-650c-4a0c-bd60-1670cffe7474-bundle\") pod \"218a3e4d-650c-4a0c-bd60-1670cffe7474\" (UID: \"218a3e4d-650c-4a0c-bd60-1670cffe7474\") "
Jul 07 17:14:53.139240 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:53.139160    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/218a3e4d-650c-4a0c-bd60-1670cffe7474-util\") pod \"218a3e4d-650c-4a0c-bd60-1670cffe7474\" (UID: \"218a3e4d-650c-4a0c-bd60-1670cffe7474\") "
Jul 07 17:14:53.139240 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:53.139187    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-prrj5\" (UniqueName: \"kubernetes.io/projected/218a3e4d-650c-4a0c-bd60-1670cffe7474-kube-api-access-prrj5\") pod \"218a3e4d-650c-4a0c-bd60-1670cffe7474\" (UID: \"218a3e4d-650c-4a0c-bd60-1670cffe7474\") "
Jul 07 17:14:53.139710 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:53.139686    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/218a3e4d-650c-4a0c-bd60-1670cffe7474-bundle" (OuterVolumeSpecName: "bundle") pod "218a3e4d-650c-4a0c-bd60-1670cffe7474" (UID: "218a3e4d-650c-4a0c-bd60-1670cffe7474"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:14:53.141237 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:53.141218    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/218a3e4d-650c-4a0c-bd60-1670cffe7474-kube-api-access-prrj5" (OuterVolumeSpecName: "kube-api-access-prrj5") pod "218a3e4d-650c-4a0c-bd60-1670cffe7474" (UID: "218a3e4d-650c-4a0c-bd60-1670cffe7474"). InnerVolumeSpecName "kube-api-access-prrj5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 17:14:53.144553 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:53.144532    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/218a3e4d-650c-4a0c-bd60-1670cffe7474-util" (OuterVolumeSpecName: "util") pod "218a3e4d-650c-4a0c-bd60-1670cffe7474" (UID: "218a3e4d-650c-4a0c-bd60-1670cffe7474"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:14:53.239928 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:53.239862    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f0112e35-a82a-43c2-9d3f-02613ca5e3e4-bundle\") pod \"f0112e35-a82a-43c2-9d3f-02613ca5e3e4\" (UID: \"f0112e35-a82a-43c2-9d3f-02613ca5e3e4\") "
Jul 07 17:14:53.239928 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:53.239924    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f0112e35-a82a-43c2-9d3f-02613ca5e3e4-util\") pod \"f0112e35-a82a-43c2-9d3f-02613ca5e3e4\" (UID: \"f0112e35-a82a-43c2-9d3f-02613ca5e3e4\") "
Jul 07 17:14:53.240089 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:53.239949    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s8tsm\" (UniqueName: \"kubernetes.io/projected/f0112e35-a82a-43c2-9d3f-02613ca5e3e4-kube-api-access-s8tsm\") pod \"f0112e35-a82a-43c2-9d3f-02613ca5e3e4\" (UID: \"f0112e35-a82a-43c2-9d3f-02613ca5e3e4\") "
Jul 07 17:14:53.240146 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:53.240134    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/218a3e4d-650c-4a0c-bd60-1670cffe7474-bundle\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:14:53.240204 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:53.240151    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/218a3e4d-650c-4a0c-bd60-1670cffe7474-util\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:14:53.240204 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:53.240164    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-prrj5\" (UniqueName: \"kubernetes.io/projected/218a3e4d-650c-4a0c-bd60-1670cffe7474-kube-api-access-prrj5\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:14:53.240496 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:53.240475    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0112e35-a82a-43c2-9d3f-02613ca5e3e4-bundle" (OuterVolumeSpecName: "bundle") pod "f0112e35-a82a-43c2-9d3f-02613ca5e3e4" (UID: "f0112e35-a82a-43c2-9d3f-02613ca5e3e4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:14:53.241964 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:53.241930    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0112e35-a82a-43c2-9d3f-02613ca5e3e4-kube-api-access-s8tsm" (OuterVolumeSpecName: "kube-api-access-s8tsm") pod "f0112e35-a82a-43c2-9d3f-02613ca5e3e4" (UID: "f0112e35-a82a-43c2-9d3f-02613ca5e3e4"). InnerVolumeSpecName "kube-api-access-s8tsm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 17:14:53.245069 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:53.245042    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0112e35-a82a-43c2-9d3f-02613ca5e3e4-util" (OuterVolumeSpecName: "util") pod "f0112e35-a82a-43c2-9d3f-02613ca5e3e4" (UID: "f0112e35-a82a-43c2-9d3f-02613ca5e3e4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:14:53.340625 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:53.340596    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f0112e35-a82a-43c2-9d3f-02613ca5e3e4-util\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:14:53.340625 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:53.340622    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s8tsm\" (UniqueName: \"kubernetes.io/projected/f0112e35-a82a-43c2-9d3f-02613ca5e3e4-kube-api-access-s8tsm\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:14:53.340625 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:53.340634    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f0112e35-a82a-43c2-9d3f-02613ca5e3e4-bundle\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:14:53.939436 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:53.939411    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvk57"
Jul 07 17:14:53.939829 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:53.939450    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvk57" event={"ID":"f0112e35-a82a-43c2-9d3f-02613ca5e3e4","Type":"ContainerDied","Data":"a1c5af041c6f62ad2f38ebdbaca7e2665ffb690e265c667c22b3d22e504b872a"}
Jul 07 17:14:53.939829 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:53.939492    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a1c5af041c6f62ad2f38ebdbaca7e2665ffb690e265c667c22b3d22e504b872a"
Jul 07 17:14:53.941288 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:53.941262    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef199k25" event={"ID":"218a3e4d-650c-4a0c-bd60-1670cffe7474","Type":"ContainerDied","Data":"361fe09e884d73cc2916f22e7aa678879c6e2a5df72d4545820cd023ec1a0a58"}
Jul 07 17:14:53.941401 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:53.941299    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="361fe09e884d73cc2916f22e7aa678879c6e2a5df72d4545820cd023ec1a0a58"
Jul 07 17:14:53.941401 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:53.941302    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef199k25"
Jul 07 17:14:55.647696 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:14:55.647663    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-dd4f44b6d-khn84"]
Jul 07 17:15:20.673449 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:20.673353    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-dd4f44b6d-khn84" podUID="3d695934-377d-4c84-8e9c-e2c9bad63689" containerName="console" containerID="cri-o://8ccfa8fa7c4e87a0aeb2d22d92803918c821934da7c10b0597bd93c96acfdfca" gracePeriod=15
Jul 07 17:15:20.903233 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:20.903206    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dd4f44b6d-khn84"
Jul 07 17:15:20.929557 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:20.929490    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3d695934-377d-4c84-8e9c-e2c9bad63689-console-oauth-config\") pod \"3d695934-377d-4c84-8e9c-e2c9bad63689\" (UID: \"3d695934-377d-4c84-8e9c-e2c9bad63689\") "
Jul 07 17:15:20.929557 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:20.929539    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3d695934-377d-4c84-8e9c-e2c9bad63689-console-config\") pod \"3d695934-377d-4c84-8e9c-e2c9bad63689\" (UID: \"3d695934-377d-4c84-8e9c-e2c9bad63689\") "
Jul 07 17:15:20.929733 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:20.929584    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4p9l8\" (UniqueName: \"kubernetes.io/projected/3d695934-377d-4c84-8e9c-e2c9bad63689-kube-api-access-4p9l8\") pod \"3d695934-377d-4c84-8e9c-e2c9bad63689\" (UID: \"3d695934-377d-4c84-8e9c-e2c9bad63689\") "
Jul 07 17:15:20.929733 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:20.929614    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3d695934-377d-4c84-8e9c-e2c9bad63689-oauth-serving-cert\") pod \"3d695934-377d-4c84-8e9c-e2c9bad63689\" (UID: \"3d695934-377d-4c84-8e9c-e2c9bad63689\") "
Jul 07 17:15:20.929733 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:20.929640    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3d695934-377d-4c84-8e9c-e2c9bad63689-trusted-ca-bundle\") pod \"3d695934-377d-4c84-8e9c-e2c9bad63689\" (UID: \"3d695934-377d-4c84-8e9c-e2c9bad63689\") "
Jul 07 17:15:20.929733 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:20.929702    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3d695934-377d-4c84-8e9c-e2c9bad63689-service-ca\") pod \"3d695934-377d-4c84-8e9c-e2c9bad63689\" (UID: \"3d695934-377d-4c84-8e9c-e2c9bad63689\") "
Jul 07 17:15:20.929733 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:20.929731    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3d695934-377d-4c84-8e9c-e2c9bad63689-console-serving-cert\") pod \"3d695934-377d-4c84-8e9c-e2c9bad63689\" (UID: \"3d695934-377d-4c84-8e9c-e2c9bad63689\") "
Jul 07 17:15:20.930097 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:20.930070    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d695934-377d-4c84-8e9c-e2c9bad63689-console-config" (OuterVolumeSpecName: "console-config") pod "3d695934-377d-4c84-8e9c-e2c9bad63689" (UID: "3d695934-377d-4c84-8e9c-e2c9bad63689"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 17:15:20.930215 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:20.930180    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d695934-377d-4c84-8e9c-e2c9bad63689-service-ca" (OuterVolumeSpecName: "service-ca") pod "3d695934-377d-4c84-8e9c-e2c9bad63689" (UID: "3d695934-377d-4c84-8e9c-e2c9bad63689"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 17:15:20.930407 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:20.930375    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d695934-377d-4c84-8e9c-e2c9bad63689-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "3d695934-377d-4c84-8e9c-e2c9bad63689" (UID: "3d695934-377d-4c84-8e9c-e2c9bad63689"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 17:15:20.930660 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:20.930628    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d695934-377d-4c84-8e9c-e2c9bad63689-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3d695934-377d-4c84-8e9c-e2c9bad63689" (UID: "3d695934-377d-4c84-8e9c-e2c9bad63689"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 17:15:20.932030 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:20.932000    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d695934-377d-4c84-8e9c-e2c9bad63689-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3d695934-377d-4c84-8e9c-e2c9bad63689" (UID: "3d695934-377d-4c84-8e9c-e2c9bad63689"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 17:15:20.932030 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:20.932021    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d695934-377d-4c84-8e9c-e2c9bad63689-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3d695934-377d-4c84-8e9c-e2c9bad63689" (UID: "3d695934-377d-4c84-8e9c-e2c9bad63689"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 17:15:20.932277 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:20.932260    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d695934-377d-4c84-8e9c-e2c9bad63689-kube-api-access-4p9l8" (OuterVolumeSpecName: "kube-api-access-4p9l8") pod "3d695934-377d-4c84-8e9c-e2c9bad63689" (UID: "3d695934-377d-4c84-8e9c-e2c9bad63689"). InnerVolumeSpecName "kube-api-access-4p9l8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 17:15:21.031046 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:21.031014    2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3d695934-377d-4c84-8e9c-e2c9bad63689-service-ca\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:15:21.031046 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:21.031035    2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3d695934-377d-4c84-8e9c-e2c9bad63689-console-serving-cert\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:15:21.031046 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:21.031044    2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3d695934-377d-4c84-8e9c-e2c9bad63689-console-oauth-config\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:15:21.031046 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:21.031053    2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3d695934-377d-4c84-8e9c-e2c9bad63689-console-config\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:15:21.031355 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:21.031062    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4p9l8\" (UniqueName: \"kubernetes.io/projected/3d695934-377d-4c84-8e9c-e2c9bad63689-kube-api-access-4p9l8\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:15:21.031355 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:21.031071    2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3d695934-377d-4c84-8e9c-e2c9bad63689-oauth-serving-cert\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:15:21.031355 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:21.031079    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3d695934-377d-4c84-8e9c-e2c9bad63689-trusted-ca-bundle\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:15:21.066616 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:21.066585    2579 generic.go:356] "Generic (PLEG): container finished" podID="3d695934-377d-4c84-8e9c-e2c9bad63689" containerID="8ccfa8fa7c4e87a0aeb2d22d92803918c821934da7c10b0597bd93c96acfdfca" exitCode=0
Jul 07 17:15:21.066710 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:21.066630    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dd4f44b6d-khn84"
Jul 07 17:15:21.066710 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:21.066675    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dd4f44b6d-khn84" event={"ID":"3d695934-377d-4c84-8e9c-e2c9bad63689","Type":"ContainerDied","Data":"8ccfa8fa7c4e87a0aeb2d22d92803918c821934da7c10b0597bd93c96acfdfca"}
Jul 07 17:15:21.066779 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:21.066718    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dd4f44b6d-khn84" event={"ID":"3d695934-377d-4c84-8e9c-e2c9bad63689","Type":"ContainerDied","Data":"3e34f674b15860e2d8df2be4e65cc5c17a65bb36a9467c9a077d13072f033410"}
Jul 07 17:15:21.066779 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:21.066739    2579 scope.go:117] "RemoveContainer" containerID="8ccfa8fa7c4e87a0aeb2d22d92803918c821934da7c10b0597bd93c96acfdfca"
Jul 07 17:15:21.075777 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:21.075761    2579 scope.go:117] "RemoveContainer" containerID="8ccfa8fa7c4e87a0aeb2d22d92803918c821934da7c10b0597bd93c96acfdfca"
Jul 07 17:15:21.076059 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:15:21.076039    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ccfa8fa7c4e87a0aeb2d22d92803918c821934da7c10b0597bd93c96acfdfca\": container with ID starting with 8ccfa8fa7c4e87a0aeb2d22d92803918c821934da7c10b0597bd93c96acfdfca not found: ID does not exist" containerID="8ccfa8fa7c4e87a0aeb2d22d92803918c821934da7c10b0597bd93c96acfdfca"
Jul 07 17:15:21.076113 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:21.076063    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ccfa8fa7c4e87a0aeb2d22d92803918c821934da7c10b0597bd93c96acfdfca"} err="failed to get container status \"8ccfa8fa7c4e87a0aeb2d22d92803918c821934da7c10b0597bd93c96acfdfca\": rpc error: code = NotFound desc = could not find container \"8ccfa8fa7c4e87a0aeb2d22d92803918c821934da7c10b0597bd93c96acfdfca\": container with ID starting with 8ccfa8fa7c4e87a0aeb2d22d92803918c821934da7c10b0597bd93c96acfdfca not found: ID does not exist"
Jul 07 17:15:21.084069 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:21.084051    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-dd4f44b6d-khn84"]
Jul 07 17:15:21.088711 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:21.088690    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-dd4f44b6d-khn84"]
Jul 07 17:15:23.027379 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:23.027348    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d695934-377d-4c84-8e9c-e2c9bad63689" path="/var/lib/kubelet/pods/3d695934-377d-4c84-8e9c-e2c9bad63689/volumes"
Jul 07 17:15:27.268391 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:27.268358    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-4cjgg"]
Jul 07 17:15:27.268988 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:27.268674    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff5c3733-6971-4414-9c3d-87d9676bea7c" containerName="pull"
Jul 07 17:15:27.268988 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:27.268685    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff5c3733-6971-4414-9c3d-87d9676bea7c" containerName="pull"
Jul 07 17:15:27.268988 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:27.268693    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="218a3e4d-650c-4a0c-bd60-1670cffe7474" containerName="pull"
Jul 07 17:15:27.268988 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:27.268699    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="218a3e4d-650c-4a0c-bd60-1670cffe7474" containerName="pull"
Jul 07 17:15:27.268988 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:27.268705    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bc00da5-eced-45fb-ac31-62874e629de7" containerName="util"
Jul 07 17:15:27.268988 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:27.268710    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bc00da5-eced-45fb-ac31-62874e629de7" containerName="util"
Jul 07 17:15:27.268988 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:27.268716    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="218a3e4d-650c-4a0c-bd60-1670cffe7474" containerName="extract"
Jul 07 17:15:27.268988 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:27.268721    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="218a3e4d-650c-4a0c-bd60-1670cffe7474" containerName="extract"
Jul 07 17:15:27.268988 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:27.268730    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff5c3733-6971-4414-9c3d-87d9676bea7c" containerName="util"
Jul 07 17:15:27.268988 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:27.268735    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff5c3733-6971-4414-9c3d-87d9676bea7c" containerName="util"
Jul 07 17:15:27.268988 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:27.268741    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0112e35-a82a-43c2-9d3f-02613ca5e3e4" containerName="pull"
Jul 07 17:15:27.268988 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:27.268746    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0112e35-a82a-43c2-9d3f-02613ca5e3e4" containerName="pull"
Jul 07 17:15:27.268988 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:27.268754    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0112e35-a82a-43c2-9d3f-02613ca5e3e4" containerName="extract"
Jul 07 17:15:27.268988 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:27.268758    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0112e35-a82a-43c2-9d3f-02613ca5e3e4" containerName="extract"
Jul 07 17:15:27.268988 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:27.268765    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d695934-377d-4c84-8e9c-e2c9bad63689" containerName="console"
Jul 07 17:15:27.268988 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:27.268769    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d695934-377d-4c84-8e9c-e2c9bad63689" containerName="console"
Jul 07 17:15:27.268988 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:27.268775    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bc00da5-eced-45fb-ac31-62874e629de7" containerName="extract"
Jul 07 17:15:27.268988 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:27.268780    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bc00da5-eced-45fb-ac31-62874e629de7" containerName="extract"
Jul 07 17:15:27.268988 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:27.268791    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="218a3e4d-650c-4a0c-bd60-1670cffe7474" containerName="util"
Jul 07 17:15:27.268988 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:27.268795    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="218a3e4d-650c-4a0c-bd60-1670cffe7474" containerName="util"
Jul 07 17:15:27.268988 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:27.268801    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bc00da5-eced-45fb-ac31-62874e629de7" containerName="pull"
Jul 07 17:15:27.268988 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:27.268806    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bc00da5-eced-45fb-ac31-62874e629de7" containerName="pull"
Jul 07 17:15:27.268988 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:27.268813    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff5c3733-6971-4414-9c3d-87d9676bea7c" containerName="extract"
Jul 07 17:15:27.268988 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:27.268818    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff5c3733-6971-4414-9c3d-87d9676bea7c" containerName="extract"
Jul 07 17:15:27.268988 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:27.268826    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0112e35-a82a-43c2-9d3f-02613ca5e3e4" containerName="util"
Jul 07 17:15:27.268988 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:27.268831    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0112e35-a82a-43c2-9d3f-02613ca5e3e4" containerName="util"
Jul 07 17:15:27.268988 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:27.268875    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d695934-377d-4c84-8e9c-e2c9bad63689" containerName="console"
Jul 07 17:15:27.268988 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:27.268882    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff5c3733-6971-4414-9c3d-87d9676bea7c" containerName="extract"
Jul 07 17:15:27.268988 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:27.268906    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="218a3e4d-650c-4a0c-bd60-1670cffe7474" containerName="extract"
Jul 07 17:15:27.268988 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:27.268914    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bc00da5-eced-45fb-ac31-62874e629de7" containerName="extract"
Jul 07 17:15:27.268988 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:27.268919    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0112e35-a82a-43c2-9d3f-02613ca5e3e4" containerName="extract"
Jul 07 17:15:27.273248 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:27.273230    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4cjgg"
Jul 07 17:15:27.275431 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:27.275412    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 07 17:15:27.275680 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:27.275666    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 07 17:15:27.276452 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:27.276435    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 07 17:15:27.276539 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:27.276464    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 07 17:15:27.276539 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:27.276435    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-vv4vg\""
Jul 07 17:15:27.282597 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:27.282573    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-4cjgg"]
Jul 07 17:15:27.378598 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:27.378568    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/7e45d93a-d792-4ba2-996a-c8ac2f28c187-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-4cjgg\" (UID: \"7e45d93a-d792-4ba2-996a-c8ac2f28c187\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4cjgg"
Jul 07 17:15:27.378770 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:27.378622    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7e45d93a-d792-4ba2-996a-c8ac2f28c187-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-4cjgg\" (UID: \"7e45d93a-d792-4ba2-996a-c8ac2f28c187\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4cjgg"
Jul 07 17:15:27.378770 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:27.378687    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6mc6\" (UniqueName: \"kubernetes.io/projected/7e45d93a-d792-4ba2-996a-c8ac2f28c187-kube-api-access-k6mc6\") pod \"kuadrant-console-plugin-66b5774985-4cjgg\" (UID: \"7e45d93a-d792-4ba2-996a-c8ac2f28c187\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4cjgg"
Jul 07 17:15:27.479682 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:27.479651    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/7e45d93a-d792-4ba2-996a-c8ac2f28c187-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-4cjgg\" (UID: \"7e45d93a-d792-4ba2-996a-c8ac2f28c187\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4cjgg"
Jul 07 17:15:27.479863 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:27.479710    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7e45d93a-d792-4ba2-996a-c8ac2f28c187-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-4cjgg\" (UID: \"7e45d93a-d792-4ba2-996a-c8ac2f28c187\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4cjgg"
Jul 07 17:15:27.479863 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:27.479839    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6mc6\" (UniqueName: \"kubernetes.io/projected/7e45d93a-d792-4ba2-996a-c8ac2f28c187-kube-api-access-k6mc6\") pod \"kuadrant-console-plugin-66b5774985-4cjgg\" (UID: \"7e45d93a-d792-4ba2-996a-c8ac2f28c187\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4cjgg"
Jul 07 17:15:27.480317 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:27.480299    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7e45d93a-d792-4ba2-996a-c8ac2f28c187-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-4cjgg\" (UID: \"7e45d93a-d792-4ba2-996a-c8ac2f28c187\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4cjgg"
Jul 07 17:15:27.482278 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:27.482257    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/7e45d93a-d792-4ba2-996a-c8ac2f28c187-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-4cjgg\" (UID: \"7e45d93a-d792-4ba2-996a-c8ac2f28c187\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4cjgg"
Jul 07 17:15:27.489707 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:27.489687    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6mc6\" (UniqueName: \"kubernetes.io/projected/7e45d93a-d792-4ba2-996a-c8ac2f28c187-kube-api-access-k6mc6\") pod \"kuadrant-console-plugin-66b5774985-4cjgg\" (UID: \"7e45d93a-d792-4ba2-996a-c8ac2f28c187\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4cjgg"
Jul 07 17:15:27.584046 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:27.583956    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4cjgg"
Jul 07 17:15:27.730493 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:27.730468    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-4cjgg"]
Jul 07 17:15:27.732851 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:15:27.732827    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7e45d93a_d792_4ba2_996a_c8ac2f28c187.slice/crio-f62836a9723c38b276b4f4a532b70b3994995a7a971a26fc4d218a63339a4123 WatchSource:0}: Error finding container f62836a9723c38b276b4f4a532b70b3994995a7a971a26fc4d218a63339a4123: Status 404 returned error can't find the container with id f62836a9723c38b276b4f4a532b70b3994995a7a971a26fc4d218a63339a4123
Jul 07 17:15:28.101790 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:28.101750    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4cjgg" event={"ID":"7e45d93a-d792-4ba2-996a-c8ac2f28c187","Type":"ContainerStarted","Data":"f62836a9723c38b276b4f4a532b70b3994995a7a971a26fc4d218a63339a4123"}
Jul 07 17:15:53.250797 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:53.250764    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4cjgg" event={"ID":"7e45d93a-d792-4ba2-996a-c8ac2f28c187","Type":"ContainerStarted","Data":"2ca40ff3ad794f2dfe0787fbb241c87009ed0fe3df590d75664dadf744bc168a"}
Jul 07 17:15:53.276354 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:15:53.276303    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4cjgg" podStartSLOduration=0.921453801 podStartE2EDuration="26.276286977s" totalImagesPullingTime="25.354833176s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:15:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:15:53.275024245 +0000 UTC m=+476.771102756" watchObservedRunningTime="2026-07-07 17:15:53.276286977 +0000 UTC m=+476.772365484"
Jul 07 17:16:13.113419 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:13.113391    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-krfj2"]
Jul 07 17:16:13.281035 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:13.281001    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-krfj2"]
Jul 07 17:16:13.281174 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:13.281093    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-krfj2"
Jul 07 17:16:13.284469 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:13.284445    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-mj2ls\""
Jul 07 17:16:13.470119 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:13.470088    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znrqb\" (UniqueName: \"kubernetes.io/projected/2297d334-8aac-4118-986f-2f161781e88f-kube-api-access-znrqb\") pod \"authorino-58cd955675-krfj2\" (UID: \"2297d334-8aac-4118-986f-2f161781e88f\") " pod="kuadrant-system/authorino-58cd955675-krfj2"
Jul 07 17:16:13.486836 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:13.486811    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-zknh9"]
Jul 07 17:16:13.502096 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:13.502076    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-zknh9"]
Jul 07 17:16:13.502195 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:13.502150    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-zknh9"
Jul 07 17:16:13.570900 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:13.570862    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znrqb\" (UniqueName: \"kubernetes.io/projected/2297d334-8aac-4118-986f-2f161781e88f-kube-api-access-znrqb\") pod \"authorino-58cd955675-krfj2\" (UID: \"2297d334-8aac-4118-986f-2f161781e88f\") " pod="kuadrant-system/authorino-58cd955675-krfj2"
Jul 07 17:16:13.580542 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:13.580521    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-znrqb\" (UniqueName: \"kubernetes.io/projected/2297d334-8aac-4118-986f-2f161781e88f-kube-api-access-znrqb\") pod \"authorino-58cd955675-krfj2\" (UID: \"2297d334-8aac-4118-986f-2f161781e88f\") " pod="kuadrant-system/authorino-58cd955675-krfj2"
Jul 07 17:16:13.590324 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:13.590291    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-krfj2"
Jul 07 17:16:13.671485 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:13.671448    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kr4tl\" (UniqueName: \"kubernetes.io/projected/2e391a1a-a2e6-4188-8091-23d0afee461e-kube-api-access-kr4tl\") pod \"authorino-75d556bd77-zknh9\" (UID: \"2e391a1a-a2e6-4188-8091-23d0afee461e\") " pod="kuadrant-system/authorino-75d556bd77-zknh9"
Jul 07 17:16:13.746475 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:13.746431    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-krfj2"]
Jul 07 17:16:13.748732 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:16:13.748698    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2297d334_8aac_4118_986f_2f161781e88f.slice/crio-8c19b7be841888c7062dbc9ad74bc342a1c9341bb0d49ae423909dcc34dc1697 WatchSource:0}: Error finding container 8c19b7be841888c7062dbc9ad74bc342a1c9341bb0d49ae423909dcc34dc1697: Status 404 returned error can't find the container with id 8c19b7be841888c7062dbc9ad74bc342a1c9341bb0d49ae423909dcc34dc1697
Jul 07 17:16:13.772810 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:13.772787    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kr4tl\" (UniqueName: \"kubernetes.io/projected/2e391a1a-a2e6-4188-8091-23d0afee461e-kube-api-access-kr4tl\") pod \"authorino-75d556bd77-zknh9\" (UID: \"2e391a1a-a2e6-4188-8091-23d0afee461e\") " pod="kuadrant-system/authorino-75d556bd77-zknh9"
Jul 07 17:16:13.782871 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:13.782847    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kr4tl\" (UniqueName: \"kubernetes.io/projected/2e391a1a-a2e6-4188-8091-23d0afee461e-kube-api-access-kr4tl\") pod \"authorino-75d556bd77-zknh9\" (UID: \"2e391a1a-a2e6-4188-8091-23d0afee461e\") " pod="kuadrant-system/authorino-75d556bd77-zknh9"
Jul 07 17:16:13.810829 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:13.810809    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-zknh9"
Jul 07 17:16:13.948553 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:13.948528    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-zknh9"]
Jul 07 17:16:13.950368 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:16:13.950333    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2e391a1a_a2e6_4188_8091_23d0afee461e.slice/crio-8f4f18b7c9d96e8b1bdda9842821676351614ca6fa924ee768d9c9e91dde72b1 WatchSource:0}: Error finding container 8f4f18b7c9d96e8b1bdda9842821676351614ca6fa924ee768d9c9e91dde72b1: Status 404 returned error can't find the container with id 8f4f18b7c9d96e8b1bdda9842821676351614ca6fa924ee768d9c9e91dde72b1
Jul 07 17:16:14.350583 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:14.350541    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-zknh9" event={"ID":"2e391a1a-a2e6-4188-8091-23d0afee461e","Type":"ContainerStarted","Data":"8f4f18b7c9d96e8b1bdda9842821676351614ca6fa924ee768d9c9e91dde72b1"}
Jul 07 17:16:14.351638 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:14.351605    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-krfj2" event={"ID":"2297d334-8aac-4118-986f-2f161781e88f","Type":"ContainerStarted","Data":"8c19b7be841888c7062dbc9ad74bc342a1c9341bb0d49ae423909dcc34dc1697"}
Jul 07 17:16:19.381050 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:19.381013    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-zknh9" event={"ID":"2e391a1a-a2e6-4188-8091-23d0afee461e","Type":"ContainerStarted","Data":"b0575ad139f7dddd384a5d489c40299e2334201882148da02aaceb3e36983c91"}
Jul 07 17:16:19.382679 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:19.382649    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-krfj2" event={"ID":"2297d334-8aac-4118-986f-2f161781e88f","Type":"ContainerStarted","Data":"512439a91ef8c59a451de6b8e89fe4180b43b3ff73cf31c3b63c7a3ba7d9a027"}
Jul 07 17:16:19.403583 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:19.403541    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-zknh9" podStartSLOduration=1.888954289 podStartE2EDuration="6.403526645s" totalImagesPullingTime="4.514572356s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:16:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:16:19.400744321 +0000 UTC m=+502.896822828" watchObservedRunningTime="2026-07-07 17:16:19.403526645 +0000 UTC m=+502.899605153"
Jul 07 17:16:19.424234 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:19.424192    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-krfj2" podStartSLOduration=1.69476937 podStartE2EDuration="6.424178703s" totalImagesPullingTime="4.729409333s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:16:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:16:19.420736815 +0000 UTC m=+502.916815325" watchObservedRunningTime="2026-07-07 17:16:19.424178703 +0000 UTC m=+502.920257211"
Jul 07 17:16:19.467201 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:19.467170    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-krfj2"]
Jul 07 17:16:21.393266 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:21.393197    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-krfj2" podUID="2297d334-8aac-4118-986f-2f161781e88f" containerName="authorino" containerID="cri-o://512439a91ef8c59a451de6b8e89fe4180b43b3ff73cf31c3b63c7a3ba7d9a027" gracePeriod=30
Jul 07 17:16:21.689174 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:21.689141    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-krfj2"
Jul 07 17:16:21.739552 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:21.739513    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-znrqb\" (UniqueName: \"kubernetes.io/projected/2297d334-8aac-4118-986f-2f161781e88f-kube-api-access-znrqb\") pod \"2297d334-8aac-4118-986f-2f161781e88f\" (UID: \"2297d334-8aac-4118-986f-2f161781e88f\") "
Jul 07 17:16:21.741673 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:21.741647    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2297d334-8aac-4118-986f-2f161781e88f-kube-api-access-znrqb" (OuterVolumeSpecName: "kube-api-access-znrqb") pod "2297d334-8aac-4118-986f-2f161781e88f" (UID: "2297d334-8aac-4118-986f-2f161781e88f"). InnerVolumeSpecName "kube-api-access-znrqb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 17:16:21.841114 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:21.841081    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-znrqb\" (UniqueName: \"kubernetes.io/projected/2297d334-8aac-4118-986f-2f161781e88f-kube-api-access-znrqb\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:16:22.399514 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:22.399471    2579 generic.go:356] "Generic (PLEG): container finished" podID="2297d334-8aac-4118-986f-2f161781e88f" containerID="512439a91ef8c59a451de6b8e89fe4180b43b3ff73cf31c3b63c7a3ba7d9a027" exitCode=0
Jul 07 17:16:22.399942 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:22.399527    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-krfj2"
Jul 07 17:16:22.399942 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:22.399554    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-krfj2" event={"ID":"2297d334-8aac-4118-986f-2f161781e88f","Type":"ContainerDied","Data":"512439a91ef8c59a451de6b8e89fe4180b43b3ff73cf31c3b63c7a3ba7d9a027"}
Jul 07 17:16:22.399942 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:22.399596    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-krfj2" event={"ID":"2297d334-8aac-4118-986f-2f161781e88f","Type":"ContainerDied","Data":"8c19b7be841888c7062dbc9ad74bc342a1c9341bb0d49ae423909dcc34dc1697"}
Jul 07 17:16:22.399942 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:22.399613    2579 scope.go:117] "RemoveContainer" containerID="512439a91ef8c59a451de6b8e89fe4180b43b3ff73cf31c3b63c7a3ba7d9a027"
Jul 07 17:16:22.408501 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:22.408480    2579 scope.go:117] "RemoveContainer" containerID="512439a91ef8c59a451de6b8e89fe4180b43b3ff73cf31c3b63c7a3ba7d9a027"
Jul 07 17:16:22.408819 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:16:22.408802    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"512439a91ef8c59a451de6b8e89fe4180b43b3ff73cf31c3b63c7a3ba7d9a027\": container with ID starting with 512439a91ef8c59a451de6b8e89fe4180b43b3ff73cf31c3b63c7a3ba7d9a027 not found: ID does not exist" containerID="512439a91ef8c59a451de6b8e89fe4180b43b3ff73cf31c3b63c7a3ba7d9a027"
Jul 07 17:16:22.408869 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:22.408830    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"512439a91ef8c59a451de6b8e89fe4180b43b3ff73cf31c3b63c7a3ba7d9a027"} err="failed to get container status \"512439a91ef8c59a451de6b8e89fe4180b43b3ff73cf31c3b63c7a3ba7d9a027\": rpc error: code = NotFound desc = could not find container \"512439a91ef8c59a451de6b8e89fe4180b43b3ff73cf31c3b63c7a3ba7d9a027\": container with ID starting with 512439a91ef8c59a451de6b8e89fe4180b43b3ff73cf31c3b63c7a3ba7d9a027 not found: ID does not exist"
Jul 07 17:16:22.422191 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:22.422157    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-krfj2"]
Jul 07 17:16:22.426581 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:22.426553    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-krfj2"]
Jul 07 17:16:23.027169 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:23.027136    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2297d334-8aac-4118-986f-2f161781e88f" path="/var/lib/kubelet/pods/2297d334-8aac-4118-986f-2f161781e88f/volumes"
Jul 07 17:16:45.761101 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:45.761070    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-tmrkp"]
Jul 07 17:16:45.761464 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:45.761368    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2297d334-8aac-4118-986f-2f161781e88f" containerName="authorino"
Jul 07 17:16:45.761464 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:45.761379    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2297d334-8aac-4118-986f-2f161781e88f" containerName="authorino"
Jul 07 17:16:45.761464 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:45.761446    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2297d334-8aac-4118-986f-2f161781e88f" containerName="authorino"
Jul 07 17:16:45.778523 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:45.778500    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-tmrkp"]
Jul 07 17:16:45.778667 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:45.778584    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-tmrkp"
Jul 07 17:16:45.824467 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:45.824443    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzxsq\" (UniqueName: \"kubernetes.io/projected/5e826f89-8b91-4f2e-bf72-8e0dea18660c-kube-api-access-wzxsq\") pod \"authorino-5d4b67c446-tmrkp\" (UID: \"5e826f89-8b91-4f2e-bf72-8e0dea18660c\") " pod="kuadrant-system/authorino-5d4b67c446-tmrkp"
Jul 07 17:16:45.925263 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:45.925235    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzxsq\" (UniqueName: \"kubernetes.io/projected/5e826f89-8b91-4f2e-bf72-8e0dea18660c-kube-api-access-wzxsq\") pod \"authorino-5d4b67c446-tmrkp\" (UID: \"5e826f89-8b91-4f2e-bf72-8e0dea18660c\") " pod="kuadrant-system/authorino-5d4b67c446-tmrkp"
Jul 07 17:16:45.934356 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:45.934330    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzxsq\" (UniqueName: \"kubernetes.io/projected/5e826f89-8b91-4f2e-bf72-8e0dea18660c-kube-api-access-wzxsq\") pod \"authorino-5d4b67c446-tmrkp\" (UID: \"5e826f89-8b91-4f2e-bf72-8e0dea18660c\") " pod="kuadrant-system/authorino-5d4b67c446-tmrkp"
Jul 07 17:16:45.971702 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:45.971676    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-tmrkp"]
Jul 07 17:16:45.971847 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:45.971836    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-tmrkp"
Jul 07 17:16:45.999933 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:45.999912    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-c8dbb7cd5-rk76j"]
Jul 07 17:16:46.003389 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:46.003373    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c8dbb7cd5-rk76j"
Jul 07 17:16:46.012811 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:46.012754    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c8dbb7cd5-rk76j"]
Jul 07 17:16:46.067126 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:46.067098    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c8dbb7cd5-rk76j"]
Jul 07 17:16:46.067365 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:16:46.067343    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-vwj87], unattached volumes=[], failed to process volumes=[kube-api-access-vwj87]: context canceled" pod="kuadrant-system/authorino-c8dbb7cd5-rk76j" podUID="de6e8ef4-fd6f-484b-aafd-5b295f591307"
Jul 07 17:16:46.105786 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:46.105758    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-586b598d45-2pkxd"]
Jul 07 17:16:46.109214 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:46.109193    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-586b598d45-2pkxd"
Jul 07 17:16:46.110546 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:46.110522    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-tmrkp"]
Jul 07 17:16:46.111536 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:46.111516    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 07 17:16:46.112492 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:16:46.112466    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5e826f89_8b91_4f2e_bf72_8e0dea18660c.slice/crio-68973bd0d20261ea188492e5402fa546705fedc29ec22478cc73f5a7a10ff0ea WatchSource:0}: Error finding container 68973bd0d20261ea188492e5402fa546705fedc29ec22478cc73f5a7a10ff0ea: Status 404 returned error can't find the container with id 68973bd0d20261ea188492e5402fa546705fedc29ec22478cc73f5a7a10ff0ea
Jul 07 17:16:46.118328 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:46.118307    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-586b598d45-2pkxd"]
Jul 07 17:16:46.127961 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:46.127939    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwj87\" (UniqueName: \"kubernetes.io/projected/de6e8ef4-fd6f-484b-aafd-5b295f591307-kube-api-access-vwj87\") pod \"authorino-c8dbb7cd5-rk76j\" (UID: \"de6e8ef4-fd6f-484b-aafd-5b295f591307\") " pod="kuadrant-system/authorino-c8dbb7cd5-rk76j"
Jul 07 17:16:46.228755 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:46.228727    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwj87\" (UniqueName: \"kubernetes.io/projected/de6e8ef4-fd6f-484b-aafd-5b295f591307-kube-api-access-vwj87\") pod \"authorino-c8dbb7cd5-rk76j\" (UID: \"de6e8ef4-fd6f-484b-aafd-5b295f591307\") " pod="kuadrant-system/authorino-c8dbb7cd5-rk76j"
Jul 07 17:16:46.228845 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:46.228807    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/286ceea6-3392-4bf9-889d-dd47a775440a-tls-cert\") pod \"authorino-586b598d45-2pkxd\" (UID: \"286ceea6-3392-4bf9-889d-dd47a775440a\") " pod="kuadrant-system/authorino-586b598d45-2pkxd"
Jul 07 17:16:46.228949 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:46.228849    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7l5dp\" (UniqueName: \"kubernetes.io/projected/286ceea6-3392-4bf9-889d-dd47a775440a-kube-api-access-7l5dp\") pod \"authorino-586b598d45-2pkxd\" (UID: \"286ceea6-3392-4bf9-889d-dd47a775440a\") " pod="kuadrant-system/authorino-586b598d45-2pkxd"
Jul 07 17:16:46.238189 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:46.238171    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwj87\" (UniqueName: \"kubernetes.io/projected/de6e8ef4-fd6f-484b-aafd-5b295f591307-kube-api-access-vwj87\") pod \"authorino-c8dbb7cd5-rk76j\" (UID: \"de6e8ef4-fd6f-484b-aafd-5b295f591307\") " pod="kuadrant-system/authorino-c8dbb7cd5-rk76j"
Jul 07 17:16:46.329545 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:46.329498    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/286ceea6-3392-4bf9-889d-dd47a775440a-tls-cert\") pod \"authorino-586b598d45-2pkxd\" (UID: \"286ceea6-3392-4bf9-889d-dd47a775440a\") " pod="kuadrant-system/authorino-586b598d45-2pkxd"
Jul 07 17:16:46.329545 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:46.329530    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l5dp\" (UniqueName: \"kubernetes.io/projected/286ceea6-3392-4bf9-889d-dd47a775440a-kube-api-access-7l5dp\") pod \"authorino-586b598d45-2pkxd\" (UID: \"286ceea6-3392-4bf9-889d-dd47a775440a\") " pod="kuadrant-system/authorino-586b598d45-2pkxd"
Jul 07 17:16:46.331695 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:46.331676    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/286ceea6-3392-4bf9-889d-dd47a775440a-tls-cert\") pod \"authorino-586b598d45-2pkxd\" (UID: \"286ceea6-3392-4bf9-889d-dd47a775440a\") " pod="kuadrant-system/authorino-586b598d45-2pkxd"
Jul 07 17:16:46.338115 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:46.338095    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7l5dp\" (UniqueName: \"kubernetes.io/projected/286ceea6-3392-4bf9-889d-dd47a775440a-kube-api-access-7l5dp\") pod \"authorino-586b598d45-2pkxd\" (UID: \"286ceea6-3392-4bf9-889d-dd47a775440a\") " pod="kuadrant-system/authorino-586b598d45-2pkxd"
Jul 07 17:16:46.427616 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:46.427593    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-586b598d45-2pkxd"
Jul 07 17:16:46.516451 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:46.516421    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-tmrkp" event={"ID":"5e826f89-8b91-4f2e-bf72-8e0dea18660c","Type":"ContainerStarted","Data":"68973bd0d20261ea188492e5402fa546705fedc29ec22478cc73f5a7a10ff0ea"}
Jul 07 17:16:46.516570 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:46.516511    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c8dbb7cd5-rk76j"
Jul 07 17:16:46.525691 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:46.525666    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c8dbb7cd5-rk76j"
Jul 07 17:16:46.560963 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:46.560943    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-586b598d45-2pkxd"]
Jul 07 17:16:46.561762 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:16:46.561742    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod286ceea6_3392_4bf9_889d_dd47a775440a.slice/crio-f968cdbdb0a96082541caed48c73cddbd1c9973787182a837c72a4f9503f4379 WatchSource:0}: Error finding container f968cdbdb0a96082541caed48c73cddbd1c9973787182a837c72a4f9503f4379: Status 404 returned error can't find the container with id f968cdbdb0a96082541caed48c73cddbd1c9973787182a837c72a4f9503f4379
Jul 07 17:16:46.631475 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:46.631406    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vwj87\" (UniqueName: \"kubernetes.io/projected/de6e8ef4-fd6f-484b-aafd-5b295f591307-kube-api-access-vwj87\") pod \"de6e8ef4-fd6f-484b-aafd-5b295f591307\" (UID: \"de6e8ef4-fd6f-484b-aafd-5b295f591307\") "
Jul 07 17:16:46.633523 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:46.633493    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de6e8ef4-fd6f-484b-aafd-5b295f591307-kube-api-access-vwj87" (OuterVolumeSpecName: "kube-api-access-vwj87") pod "de6e8ef4-fd6f-484b-aafd-5b295f591307" (UID: "de6e8ef4-fd6f-484b-aafd-5b295f591307"). InnerVolumeSpecName "kube-api-access-vwj87". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 17:16:46.732662 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:46.732635    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vwj87\" (UniqueName: \"kubernetes.io/projected/de6e8ef4-fd6f-484b-aafd-5b295f591307-kube-api-access-vwj87\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:16:47.522594 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:47.522509    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-tmrkp" event={"ID":"5e826f89-8b91-4f2e-bf72-8e0dea18660c","Type":"ContainerStarted","Data":"548d2f588224a3ba9440a475418f31ebf36b5170c6c56fe4f7abbfa0bfd45aa7"}
Jul 07 17:16:47.522594 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:47.522541    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-tmrkp" podUID="5e826f89-8b91-4f2e-bf72-8e0dea18660c" containerName="authorino" containerID="cri-o://548d2f588224a3ba9440a475418f31ebf36b5170c6c56fe4f7abbfa0bfd45aa7" gracePeriod=30
Jul 07 17:16:47.524301 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:47.524280    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c8dbb7cd5-rk76j"
Jul 07 17:16:47.524432 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:47.524313    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-586b598d45-2pkxd" event={"ID":"286ceea6-3392-4bf9-889d-dd47a775440a","Type":"ContainerStarted","Data":"501f5eaddd31c3e255e39c59c61c7cb719b8e4559dc8c2947825d75fe63e318b"}
Jul 07 17:16:47.524432 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:47.524335    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-586b598d45-2pkxd" event={"ID":"286ceea6-3392-4bf9-889d-dd47a775440a","Type":"ContainerStarted","Data":"f968cdbdb0a96082541caed48c73cddbd1c9973787182a837c72a4f9503f4379"}
Jul 07 17:16:47.559697 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:47.559655    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-tmrkp" podStartSLOduration=2.16590671 podStartE2EDuration="2.559637943s" totalImagesPullingTime="393.731233ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:16:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:16:47.549021026 +0000 UTC m=+531.045099536" watchObservedRunningTime="2026-07-07 17:16:47.559637943 +0000 UTC m=+531.055716453"
Jul 07 17:16:47.600438 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:47.600405    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c8dbb7cd5-rk76j"]
Jul 07 17:16:47.602945 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:47.602916    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-c8dbb7cd5-rk76j"]
Jul 07 17:16:47.619845 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:47.619804    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-586b598d45-2pkxd" podStartSLOduration=1.2015092840000001 podStartE2EDuration="1.619791765s" totalImagesPullingTime="418.282481ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:16:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:16:47.619599098 +0000 UTC m=+531.115677605" watchObservedRunningTime="2026-07-07 17:16:47.619791765 +0000 UTC m=+531.115870272"
Jul 07 17:16:47.665061 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:47.665031    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-zknh9"]
Jul 07 17:16:47.665255 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:47.665218    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-zknh9" podUID="2e391a1a-a2e6-4188-8091-23d0afee461e" containerName="authorino" containerID="cri-o://b0575ad139f7dddd384a5d489c40299e2334201882148da02aaceb3e36983c91" gracePeriod=30
Jul 07 17:16:47.777122 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:47.776976    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-tmrkp"
Jul 07 17:16:47.843478 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:47.843452    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wzxsq\" (UniqueName: \"kubernetes.io/projected/5e826f89-8b91-4f2e-bf72-8e0dea18660c-kube-api-access-wzxsq\") pod \"5e826f89-8b91-4f2e-bf72-8e0dea18660c\" (UID: \"5e826f89-8b91-4f2e-bf72-8e0dea18660c\") "
Jul 07 17:16:47.845586 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:47.845556    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e826f89-8b91-4f2e-bf72-8e0dea18660c-kube-api-access-wzxsq" (OuterVolumeSpecName: "kube-api-access-wzxsq") pod "5e826f89-8b91-4f2e-bf72-8e0dea18660c" (UID: "5e826f89-8b91-4f2e-bf72-8e0dea18660c"). InnerVolumeSpecName "kube-api-access-wzxsq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 17:16:47.892166 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:47.892141    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-zknh9"
Jul 07 17:16:47.944260 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:47.944238    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kr4tl\" (UniqueName: \"kubernetes.io/projected/2e391a1a-a2e6-4188-8091-23d0afee461e-kube-api-access-kr4tl\") pod \"2e391a1a-a2e6-4188-8091-23d0afee461e\" (UID: \"2e391a1a-a2e6-4188-8091-23d0afee461e\") "
Jul 07 17:16:47.944390 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:47.944379    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wzxsq\" (UniqueName: \"kubernetes.io/projected/5e826f89-8b91-4f2e-bf72-8e0dea18660c-kube-api-access-wzxsq\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:16:47.946359 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:47.946334    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e391a1a-a2e6-4188-8091-23d0afee461e-kube-api-access-kr4tl" (OuterVolumeSpecName: "kube-api-access-kr4tl") pod "2e391a1a-a2e6-4188-8091-23d0afee461e" (UID: "2e391a1a-a2e6-4188-8091-23d0afee461e"). InnerVolumeSpecName "kube-api-access-kr4tl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 17:16:48.045286 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:48.045225    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kr4tl\" (UniqueName: \"kubernetes.io/projected/2e391a1a-a2e6-4188-8091-23d0afee461e-kube-api-access-kr4tl\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:16:48.530412 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:48.530376    2579 generic.go:356] "Generic (PLEG): container finished" podID="2e391a1a-a2e6-4188-8091-23d0afee461e" containerID="b0575ad139f7dddd384a5d489c40299e2334201882148da02aaceb3e36983c91" exitCode=0
Jul 07 17:16:48.530831 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:48.530419    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-zknh9"
Jul 07 17:16:48.530831 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:48.530469    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-zknh9" event={"ID":"2e391a1a-a2e6-4188-8091-23d0afee461e","Type":"ContainerDied","Data":"b0575ad139f7dddd384a5d489c40299e2334201882148da02aaceb3e36983c91"}
Jul 07 17:16:48.530831 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:48.530507    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-zknh9" event={"ID":"2e391a1a-a2e6-4188-8091-23d0afee461e","Type":"ContainerDied","Data":"8f4f18b7c9d96e8b1bdda9842821676351614ca6fa924ee768d9c9e91dde72b1"}
Jul 07 17:16:48.530831 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:48.530523    2579 scope.go:117] "RemoveContainer" containerID="b0575ad139f7dddd384a5d489c40299e2334201882148da02aaceb3e36983c91"
Jul 07 17:16:48.531881 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:48.531855    2579 generic.go:356] "Generic (PLEG): container finished" podID="5e826f89-8b91-4f2e-bf72-8e0dea18660c" containerID="548d2f588224a3ba9440a475418f31ebf36b5170c6c56fe4f7abbfa0bfd45aa7" exitCode=0
Jul 07 17:16:48.532031 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:48.531924    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-tmrkp"
Jul 07 17:16:48.532031 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:48.531929    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-tmrkp" event={"ID":"5e826f89-8b91-4f2e-bf72-8e0dea18660c","Type":"ContainerDied","Data":"548d2f588224a3ba9440a475418f31ebf36b5170c6c56fe4f7abbfa0bfd45aa7"}
Jul 07 17:16:48.532031 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:48.531974    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-tmrkp" event={"ID":"5e826f89-8b91-4f2e-bf72-8e0dea18660c","Type":"ContainerDied","Data":"68973bd0d20261ea188492e5402fa546705fedc29ec22478cc73f5a7a10ff0ea"}
Jul 07 17:16:48.542601 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:48.542391    2579 scope.go:117] "RemoveContainer" containerID="b0575ad139f7dddd384a5d489c40299e2334201882148da02aaceb3e36983c91"
Jul 07 17:16:48.543030 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:16:48.543011    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0575ad139f7dddd384a5d489c40299e2334201882148da02aaceb3e36983c91\": container with ID starting with b0575ad139f7dddd384a5d489c40299e2334201882148da02aaceb3e36983c91 not found: ID does not exist" containerID="b0575ad139f7dddd384a5d489c40299e2334201882148da02aaceb3e36983c91"
Jul 07 17:16:48.543123 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:48.543036    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0575ad139f7dddd384a5d489c40299e2334201882148da02aaceb3e36983c91"} err="failed to get container status \"b0575ad139f7dddd384a5d489c40299e2334201882148da02aaceb3e36983c91\": rpc error: code = NotFound desc = could not find container \"b0575ad139f7dddd384a5d489c40299e2334201882148da02aaceb3e36983c91\": container with ID starting with b0575ad139f7dddd384a5d489c40299e2334201882148da02aaceb3e36983c91 not found: ID does not exist"
Jul 07 17:16:48.543123 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:48.543051    2579 scope.go:117] "RemoveContainer" containerID="548d2f588224a3ba9440a475418f31ebf36b5170c6c56fe4f7abbfa0bfd45aa7"
Jul 07 17:16:48.550715 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:48.550700    2579 scope.go:117] "RemoveContainer" containerID="548d2f588224a3ba9440a475418f31ebf36b5170c6c56fe4f7abbfa0bfd45aa7"
Jul 07 17:16:48.550967 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:16:48.550951    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"548d2f588224a3ba9440a475418f31ebf36b5170c6c56fe4f7abbfa0bfd45aa7\": container with ID starting with 548d2f588224a3ba9440a475418f31ebf36b5170c6c56fe4f7abbfa0bfd45aa7 not found: ID does not exist" containerID="548d2f588224a3ba9440a475418f31ebf36b5170c6c56fe4f7abbfa0bfd45aa7"
Jul 07 17:16:48.551021 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:48.550974    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"548d2f588224a3ba9440a475418f31ebf36b5170c6c56fe4f7abbfa0bfd45aa7"} err="failed to get container status \"548d2f588224a3ba9440a475418f31ebf36b5170c6c56fe4f7abbfa0bfd45aa7\": rpc error: code = NotFound desc = could not find container \"548d2f588224a3ba9440a475418f31ebf36b5170c6c56fe4f7abbfa0bfd45aa7\": container with ID starting with 548d2f588224a3ba9440a475418f31ebf36b5170c6c56fe4f7abbfa0bfd45aa7 not found: ID does not exist"
Jul 07 17:16:48.557853 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:48.557834    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-tmrkp"]
Jul 07 17:16:48.561086 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:48.561069    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-tmrkp"]
Jul 07 17:16:48.574386 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:48.574333    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-zknh9"]
Jul 07 17:16:48.576138 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:48.576117    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-zknh9"]
Jul 07 17:16:49.027410 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:49.027380    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e391a1a-a2e6-4188-8091-23d0afee461e" path="/var/lib/kubelet/pods/2e391a1a-a2e6-4188-8091-23d0afee461e/volumes"
Jul 07 17:16:49.027762 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:49.027745    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e826f89-8b91-4f2e-bf72-8e0dea18660c" path="/var/lib/kubelet/pods/5e826f89-8b91-4f2e-bf72-8e0dea18660c/volumes"
Jul 07 17:16:49.028110 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:16:49.028092    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de6e8ef4-fd6f-484b-aafd-5b295f591307" path="/var/lib/kubelet/pods/de6e8ef4-fd6f-484b-aafd-5b295f591307/volumes"
Jul 07 17:18:20.580278 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:20.580149    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-ftlqk"]
Jul 07 17:18:20.582494 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:20.580622    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e391a1a-a2e6-4188-8091-23d0afee461e" containerName="authorino"
Jul 07 17:18:20.582494 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:20.580640    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e391a1a-a2e6-4188-8091-23d0afee461e" containerName="authorino"
Jul 07 17:18:20.582494 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:20.580653    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e826f89-8b91-4f2e-bf72-8e0dea18660c" containerName="authorino"
Jul 07 17:18:20.582494 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:20.580662    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e826f89-8b91-4f2e-bf72-8e0dea18660c" containerName="authorino"
Jul 07 17:18:20.582494 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:20.580723    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e391a1a-a2e6-4188-8091-23d0afee461e" containerName="authorino"
Jul 07 17:18:20.582494 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:20.580734    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e826f89-8b91-4f2e-bf72-8e0dea18660c" containerName="authorino"
Jul 07 17:18:20.583422 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:20.583405    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ftlqk"
Jul 07 17:18:20.586594 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:20.586570    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 07 17:18:20.586711 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:20.586616    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 07 17:18:20.586711 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:20.586691    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-hltvx\""
Jul 07 17:18:20.586850 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:20.586835    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 07 17:18:20.595736 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:20.595714    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-ftlqk"]
Jul 07 17:18:20.640697 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:20.640670    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c0c92c6-9968-43b2-97d9-e26e36415dbd-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ftlqk\" (UID: \"8c0c92c6-9968-43b2-97d9-e26e36415dbd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ftlqk"
Jul 07 17:18:20.640783 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:20.640705    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c0c92c6-9968-43b2-97d9-e26e36415dbd-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ftlqk\" (UID: \"8c0c92c6-9968-43b2-97d9-e26e36415dbd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ftlqk"
Jul 07 17:18:20.640783 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:20.640723    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c0c92c6-9968-43b2-97d9-e26e36415dbd-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ftlqk\" (UID: \"8c0c92c6-9968-43b2-97d9-e26e36415dbd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ftlqk"
Jul 07 17:18:20.640858 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:20.640802    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c0c92c6-9968-43b2-97d9-e26e36415dbd-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ftlqk\" (UID: \"8c0c92c6-9968-43b2-97d9-e26e36415dbd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ftlqk"
Jul 07 17:18:20.640858 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:20.640840    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c0c92c6-9968-43b2-97d9-e26e36415dbd-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ftlqk\" (UID: \"8c0c92c6-9968-43b2-97d9-e26e36415dbd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ftlqk"
Jul 07 17:18:20.640944 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:20.640862    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5h6mj\" (UniqueName: \"kubernetes.io/projected/8c0c92c6-9968-43b2-97d9-e26e36415dbd-kube-api-access-5h6mj\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ftlqk\" (UID: \"8c0c92c6-9968-43b2-97d9-e26e36415dbd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ftlqk"
Jul 07 17:18:20.741997 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:20.741969    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c0c92c6-9968-43b2-97d9-e26e36415dbd-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ftlqk\" (UID: \"8c0c92c6-9968-43b2-97d9-e26e36415dbd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ftlqk"
Jul 07 17:18:20.742091 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:20.742003    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5h6mj\" (UniqueName: \"kubernetes.io/projected/8c0c92c6-9968-43b2-97d9-e26e36415dbd-kube-api-access-5h6mj\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ftlqk\" (UID: \"8c0c92c6-9968-43b2-97d9-e26e36415dbd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ftlqk"
Jul 07 17:18:20.742091 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:20.742045    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c0c92c6-9968-43b2-97d9-e26e36415dbd-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ftlqk\" (UID: \"8c0c92c6-9968-43b2-97d9-e26e36415dbd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ftlqk"
Jul 07 17:18:20.742091 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:20.742084    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c0c92c6-9968-43b2-97d9-e26e36415dbd-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ftlqk\" (UID: \"8c0c92c6-9968-43b2-97d9-e26e36415dbd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ftlqk"
Jul 07 17:18:20.742220 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:20.742107    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c0c92c6-9968-43b2-97d9-e26e36415dbd-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ftlqk\" (UID: \"8c0c92c6-9968-43b2-97d9-e26e36415dbd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ftlqk"
Jul 07 17:18:20.742220 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:20.742167    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c0c92c6-9968-43b2-97d9-e26e36415dbd-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ftlqk\" (UID: \"8c0c92c6-9968-43b2-97d9-e26e36415dbd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ftlqk"
Jul 07 17:18:20.742454 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:20.742436    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c0c92c6-9968-43b2-97d9-e26e36415dbd-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ftlqk\" (UID: \"8c0c92c6-9968-43b2-97d9-e26e36415dbd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ftlqk"
Jul 07 17:18:20.742522 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:20.742481    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c0c92c6-9968-43b2-97d9-e26e36415dbd-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ftlqk\" (UID: \"8c0c92c6-9968-43b2-97d9-e26e36415dbd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ftlqk"
Jul 07 17:18:20.742581 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:20.742538    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c0c92c6-9968-43b2-97d9-e26e36415dbd-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ftlqk\" (UID: \"8c0c92c6-9968-43b2-97d9-e26e36415dbd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ftlqk"
Jul 07 17:18:20.744327 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:20.744308    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c0c92c6-9968-43b2-97d9-e26e36415dbd-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ftlqk\" (UID: \"8c0c92c6-9968-43b2-97d9-e26e36415dbd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ftlqk"
Jul 07 17:18:20.744564 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:20.744547    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c0c92c6-9968-43b2-97d9-e26e36415dbd-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ftlqk\" (UID: \"8c0c92c6-9968-43b2-97d9-e26e36415dbd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ftlqk"
Jul 07 17:18:20.751110 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:20.751092    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5h6mj\" (UniqueName: \"kubernetes.io/projected/8c0c92c6-9968-43b2-97d9-e26e36415dbd-kube-api-access-5h6mj\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ftlqk\" (UID: \"8c0c92c6-9968-43b2-97d9-e26e36415dbd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ftlqk"
Jul 07 17:18:20.894136 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:20.894110    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ftlqk"
Jul 07 17:18:21.025825 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:18:21.025794    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c0c92c6_9968_43b2_97d9_e26e36415dbd.slice/crio-1b7d03e91403aa8e038e926270787abedb6835619ed053634e386af3bae5a9dc WatchSource:0}: Error finding container 1b7d03e91403aa8e038e926270787abedb6835619ed053634e386af3bae5a9dc: Status 404 returned error can't find the container with id 1b7d03e91403aa8e038e926270787abedb6835619ed053634e386af3bae5a9dc
Jul 07 17:18:21.027275 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:21.027256    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 17:18:21.028813 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:21.028790    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-ftlqk"]
Jul 07 17:18:22.008311 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:22.008272    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ftlqk" event={"ID":"8c0c92c6-9968-43b2-97d9-e26e36415dbd","Type":"ContainerStarted","Data":"1b7d03e91403aa8e038e926270787abedb6835619ed053634e386af3bae5a9dc"}
Jul 07 17:18:23.015381 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:23.015347    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ftlqk" event={"ID":"8c0c92c6-9968-43b2-97d9-e26e36415dbd","Type":"ContainerStarted","Data":"a448991e0fdda641f820f9997abb2b13725418f7801b8de3dcc7c0550451e30b"}
Jul 07 17:18:23.037198 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:23.037143    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ftlqk" podStartSLOduration=1.894679457 podStartE2EDuration="3.037126806s" totalImagesPullingTime="1.142447349s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:18:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:18:23.034861112 +0000 UTC m=+626.530939620" watchObservedRunningTime="2026-07-07 17:18:23.037126806 +0000 UTC m=+626.533205314"
Jul 07 17:18:30.894802 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:30.894761    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ftlqk"
Jul 07 17:18:30.895215 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:30.894831    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ftlqk"
Jul 07 17:18:30.907685 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:30.907659    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ftlqk"
Jul 07 17:18:31.066134 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:31.066105    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ftlqk"
Jul 07 17:18:38.475399 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:38.475362    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvm6bt"]
Jul 07 17:18:38.478488 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:38.478470    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvm6bt"
Jul 07 17:18:38.480669 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:38.480648    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 07 17:18:38.489127 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:38.489106    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvm6bt"]
Jul 07 17:18:38.598067 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:38.598034    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phxt9\" (UniqueName: \"kubernetes.io/projected/268f092d-b8cd-4aa9-8c79-e28b19312630-kube-api-access-phxt9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvm6bt\" (UID: \"268f092d-b8cd-4aa9-8c79-e28b19312630\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvm6bt"
Jul 07 17:18:38.598236 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:38.598078    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/268f092d-b8cd-4aa9-8c79-e28b19312630-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvm6bt\" (UID: \"268f092d-b8cd-4aa9-8c79-e28b19312630\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvm6bt"
Jul 07 17:18:38.598236 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:38.598161    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/268f092d-b8cd-4aa9-8c79-e28b19312630-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvm6bt\" (UID: \"268f092d-b8cd-4aa9-8c79-e28b19312630\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvm6bt"
Jul 07 17:18:38.598236 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:38.598188    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/268f092d-b8cd-4aa9-8c79-e28b19312630-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvm6bt\" (UID: \"268f092d-b8cd-4aa9-8c79-e28b19312630\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvm6bt"
Jul 07 17:18:38.598361 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:38.598252    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/268f092d-b8cd-4aa9-8c79-e28b19312630-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvm6bt\" (UID: \"268f092d-b8cd-4aa9-8c79-e28b19312630\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvm6bt"
Jul 07 17:18:38.598361 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:38.598283    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/268f092d-b8cd-4aa9-8c79-e28b19312630-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvm6bt\" (UID: \"268f092d-b8cd-4aa9-8c79-e28b19312630\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvm6bt"
Jul 07 17:18:38.699250 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:38.699210    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/268f092d-b8cd-4aa9-8c79-e28b19312630-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvm6bt\" (UID: \"268f092d-b8cd-4aa9-8c79-e28b19312630\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvm6bt"
Jul 07 17:18:38.699425 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:38.699270    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/268f092d-b8cd-4aa9-8c79-e28b19312630-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvm6bt\" (UID: \"268f092d-b8cd-4aa9-8c79-e28b19312630\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvm6bt"
Jul 07 17:18:38.699425 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:38.699295    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/268f092d-b8cd-4aa9-8c79-e28b19312630-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvm6bt\" (UID: \"268f092d-b8cd-4aa9-8c79-e28b19312630\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvm6bt"
Jul 07 17:18:38.699425 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:38.699334    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/268f092d-b8cd-4aa9-8c79-e28b19312630-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvm6bt\" (UID: \"268f092d-b8cd-4aa9-8c79-e28b19312630\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvm6bt"
Jul 07 17:18:38.699425 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:38.699356    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/268f092d-b8cd-4aa9-8c79-e28b19312630-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvm6bt\" (UID: \"268f092d-b8cd-4aa9-8c79-e28b19312630\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvm6bt"
Jul 07 17:18:38.699425 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:38.699375    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phxt9\" (UniqueName: \"kubernetes.io/projected/268f092d-b8cd-4aa9-8c79-e28b19312630-kube-api-access-phxt9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvm6bt\" (UID: \"268f092d-b8cd-4aa9-8c79-e28b19312630\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvm6bt"
Jul 07 17:18:38.699777 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:38.699752    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/268f092d-b8cd-4aa9-8c79-e28b19312630-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvm6bt\" (UID: \"268f092d-b8cd-4aa9-8c79-e28b19312630\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvm6bt"
Jul 07 17:18:38.699837 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:38.699797    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/268f092d-b8cd-4aa9-8c79-e28b19312630-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvm6bt\" (UID: \"268f092d-b8cd-4aa9-8c79-e28b19312630\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvm6bt"
Jul 07 17:18:38.699917 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:38.699866    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/268f092d-b8cd-4aa9-8c79-e28b19312630-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvm6bt\" (UID: \"268f092d-b8cd-4aa9-8c79-e28b19312630\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvm6bt"
Jul 07 17:18:38.701717 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:38.701691    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/268f092d-b8cd-4aa9-8c79-e28b19312630-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvm6bt\" (UID: \"268f092d-b8cd-4aa9-8c79-e28b19312630\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvm6bt"
Jul 07 17:18:38.702061 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:38.702039    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/268f092d-b8cd-4aa9-8c79-e28b19312630-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvm6bt\" (UID: \"268f092d-b8cd-4aa9-8c79-e28b19312630\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvm6bt"
Jul 07 17:18:38.708307 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:38.708286    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-phxt9\" (UniqueName: \"kubernetes.io/projected/268f092d-b8cd-4aa9-8c79-e28b19312630-kube-api-access-phxt9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvm6bt\" (UID: \"268f092d-b8cd-4aa9-8c79-e28b19312630\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvm6bt"
Jul 07 17:18:38.789502 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:38.789405    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvm6bt"
Jul 07 17:18:38.929365 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:38.929334    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvm6bt"]
Jul 07 17:18:38.930462 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:18:38.930438    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod268f092d_b8cd_4aa9_8c79_e28b19312630.slice/crio-e7403fe8877532fe3c3f22f29b81d8d22cc8c934e1be879f43a9f5829d40ac11 WatchSource:0}: Error finding container e7403fe8877532fe3c3f22f29b81d8d22cc8c934e1be879f43a9f5829d40ac11: Status 404 returned error can't find the container with id e7403fe8877532fe3c3f22f29b81d8d22cc8c934e1be879f43a9f5829d40ac11
Jul 07 17:18:39.093985 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:39.093906    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvm6bt" event={"ID":"268f092d-b8cd-4aa9-8c79-e28b19312630","Type":"ContainerStarted","Data":"e7403fe8877532fe3c3f22f29b81d8d22cc8c934e1be879f43a9f5829d40ac11"}
Jul 07 17:18:40.100584 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:40.100549    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvm6bt" event={"ID":"268f092d-b8cd-4aa9-8c79-e28b19312630","Type":"ContainerStarted","Data":"d20423c016343a5bb8dbbf1ecbacc7b07bfbb4b0da25390d0668f99d3b6695ec"}
Jul 07 17:18:40.123904 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:40.123849    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvm6bt" podStartSLOduration=1.9439112170000001 podStartE2EDuration="2.12383463s" totalImagesPullingTime="179.923413ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:18:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:18:40.12135327 +0000 UTC m=+643.617431781" watchObservedRunningTime="2026-07-07 17:18:40.12383463 +0000 UTC m=+643.619913139"
Jul 07 17:18:48.789735 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:48.789697    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvm6bt"
Jul 07 17:18:48.790124 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:48.789791    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvm6bt"
Jul 07 17:18:48.801922 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:48.801900    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvm6bt"
Jul 07 17:18:49.161678 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:49.161653    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvm6bt"
Jul 07 17:18:58.479173 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:58.479132    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75lgx"]
Jul 07 17:18:58.488358 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:58.488321    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75lgx"
Jul 07 17:18:58.490834 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:58.490814    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 07 17:18:58.492342 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:58.492292    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75lgx"]
Jul 07 17:18:58.572862 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:58.572827    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/90a24b15-407c-43f6-9d33-d3ab01909153-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-75lgx\" (UID: \"90a24b15-407c-43f6-9d33-d3ab01909153\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75lgx"
Jul 07 17:18:58.573075 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:58.572874    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/90a24b15-407c-43f6-9d33-d3ab01909153-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-75lgx\" (UID: \"90a24b15-407c-43f6-9d33-d3ab01909153\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75lgx"
Jul 07 17:18:58.573075 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:58.572959    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9jk9\" (UniqueName: \"kubernetes.io/projected/90a24b15-407c-43f6-9d33-d3ab01909153-kube-api-access-m9jk9\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-75lgx\" (UID: \"90a24b15-407c-43f6-9d33-d3ab01909153\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75lgx"
Jul 07 17:18:58.573075 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:58.572996    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90a24b15-407c-43f6-9d33-d3ab01909153-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-75lgx\" (UID: \"90a24b15-407c-43f6-9d33-d3ab01909153\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75lgx"
Jul 07 17:18:58.573075 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:58.573025    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/90a24b15-407c-43f6-9d33-d3ab01909153-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-75lgx\" (UID: \"90a24b15-407c-43f6-9d33-d3ab01909153\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75lgx"
Jul 07 17:18:58.573257 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:58.573125    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90a24b15-407c-43f6-9d33-d3ab01909153-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-75lgx\" (UID: \"90a24b15-407c-43f6-9d33-d3ab01909153\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75lgx"
Jul 07 17:18:58.673687 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:58.673644    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/90a24b15-407c-43f6-9d33-d3ab01909153-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-75lgx\" (UID: \"90a24b15-407c-43f6-9d33-d3ab01909153\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75lgx"
Jul 07 17:18:58.673834 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:58.673692    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/90a24b15-407c-43f6-9d33-d3ab01909153-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-75lgx\" (UID: \"90a24b15-407c-43f6-9d33-d3ab01909153\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75lgx"
Jul 07 17:18:58.673834 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:58.673719    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9jk9\" (UniqueName: \"kubernetes.io/projected/90a24b15-407c-43f6-9d33-d3ab01909153-kube-api-access-m9jk9\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-75lgx\" (UID: \"90a24b15-407c-43f6-9d33-d3ab01909153\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75lgx"
Jul 07 17:18:58.673834 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:58.673743    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90a24b15-407c-43f6-9d33-d3ab01909153-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-75lgx\" (UID: \"90a24b15-407c-43f6-9d33-d3ab01909153\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75lgx"
Jul 07 17:18:58.673834 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:58.673762    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/90a24b15-407c-43f6-9d33-d3ab01909153-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-75lgx\" (UID: \"90a24b15-407c-43f6-9d33-d3ab01909153\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75lgx"
Jul 07 17:18:58.674081 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:58.673952    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90a24b15-407c-43f6-9d33-d3ab01909153-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-75lgx\" (UID: \"90a24b15-407c-43f6-9d33-d3ab01909153\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75lgx"
Jul 07 17:18:58.674139 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:58.674112    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/90a24b15-407c-43f6-9d33-d3ab01909153-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-75lgx\" (UID: \"90a24b15-407c-43f6-9d33-d3ab01909153\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75lgx"
Jul 07 17:18:58.674216 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:58.674194    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/90a24b15-407c-43f6-9d33-d3ab01909153-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-75lgx\" (UID: \"90a24b15-407c-43f6-9d33-d3ab01909153\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75lgx"
Jul 07 17:18:58.674280 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:58.674266    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90a24b15-407c-43f6-9d33-d3ab01909153-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-75lgx\" (UID: \"90a24b15-407c-43f6-9d33-d3ab01909153\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75lgx"
Jul 07 17:18:58.676137 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:58.676112    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/90a24b15-407c-43f6-9d33-d3ab01909153-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-75lgx\" (UID: \"90a24b15-407c-43f6-9d33-d3ab01909153\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75lgx"
Jul 07 17:18:58.676400 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:58.676382    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90a24b15-407c-43f6-9d33-d3ab01909153-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-75lgx\" (UID: \"90a24b15-407c-43f6-9d33-d3ab01909153\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75lgx"
Jul 07 17:18:58.686671 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:58.686647    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9jk9\" (UniqueName: \"kubernetes.io/projected/90a24b15-407c-43f6-9d33-d3ab01909153-kube-api-access-m9jk9\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-75lgx\" (UID: \"90a24b15-407c-43f6-9d33-d3ab01909153\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75lgx"
Jul 07 17:18:58.799114 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:58.799023    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75lgx"
Jul 07 17:18:58.935699 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:58.935674    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75lgx"]
Jul 07 17:18:58.937682 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:18:58.937656    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod90a24b15_407c_43f6_9d33_d3ab01909153.slice/crio-f2ade7b85bdbd2ae814f0ca82ef668fbb811d27f7649d78507c7425ec18db30f WatchSource:0}: Error finding container f2ade7b85bdbd2ae814f0ca82ef668fbb811d27f7649d78507c7425ec18db30f: Status 404 returned error can't find the container with id f2ade7b85bdbd2ae814f0ca82ef668fbb811d27f7649d78507c7425ec18db30f
Jul 07 17:18:59.199683 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:18:59.199646    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75lgx" event={"ID":"90a24b15-407c-43f6-9d33-d3ab01909153","Type":"ContainerStarted","Data":"f2ade7b85bdbd2ae814f0ca82ef668fbb811d27f7649d78507c7425ec18db30f"}
Jul 07 17:19:00.206833 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:19:00.206802    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75lgx" event={"ID":"90a24b15-407c-43f6-9d33-d3ab01909153","Type":"ContainerStarted","Data":"66b38bc33dceaa55d254c1db64fa9e0ea8a76e6da02d9b6755e2700aaaac79e4"}
Jul 07 17:19:00.231300 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:19:00.231253    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75lgx" podStartSLOduration=2.001212436 podStartE2EDuration="2.231237798s" totalImagesPullingTime="230.025362ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:18:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:19:00.228832206 +0000 UTC m=+663.724910714" watchObservedRunningTime="2026-07-07 17:19:00.231237798 +0000 UTC m=+663.727316306"
Jul 07 17:19:08.800059 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:19:08.800029    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75lgx"
Jul 07 17:19:08.800497 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:19:08.800079    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75lgx"
Jul 07 17:19:08.812130 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:19:08.812111    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75lgx"
Jul 07 17:19:09.261255 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:19:09.261224    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-75lgx"
Jul 07 17:20:03.071256 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:20:03.071183    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5c4697d5b4-9rpkm"]
Jul 07 17:20:03.074426 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:20:03.074408    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5c4697d5b4-9rpkm"
Jul 07 17:20:03.084862 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:20:03.084839    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5c4697d5b4-9rpkm"]
Jul 07 17:20:03.141679 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:20:03.141653    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bz92n\" (UniqueName: \"kubernetes.io/projected/4724ce74-14df-4bf5-a303-ba98f71b6ebe-kube-api-access-bz92n\") pod \"authorino-5c4697d5b4-9rpkm\" (UID: \"4724ce74-14df-4bf5-a303-ba98f71b6ebe\") " pod="kuadrant-system/authorino-5c4697d5b4-9rpkm"
Jul 07 17:20:03.142000 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:20:03.141699    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4724ce74-14df-4bf5-a303-ba98f71b6ebe-tls-cert\") pod \"authorino-5c4697d5b4-9rpkm\" (UID: \"4724ce74-14df-4bf5-a303-ba98f71b6ebe\") " pod="kuadrant-system/authorino-5c4697d5b4-9rpkm"
Jul 07 17:20:03.242857 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:20:03.242828    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bz92n\" (UniqueName: \"kubernetes.io/projected/4724ce74-14df-4bf5-a303-ba98f71b6ebe-kube-api-access-bz92n\") pod \"authorino-5c4697d5b4-9rpkm\" (UID: \"4724ce74-14df-4bf5-a303-ba98f71b6ebe\") " pod="kuadrant-system/authorino-5c4697d5b4-9rpkm"
Jul 07 17:20:03.242991 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:20:03.242912    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4724ce74-14df-4bf5-a303-ba98f71b6ebe-tls-cert\") pod \"authorino-5c4697d5b4-9rpkm\" (UID: \"4724ce74-14df-4bf5-a303-ba98f71b6ebe\") " pod="kuadrant-system/authorino-5c4697d5b4-9rpkm"
Jul 07 17:20:03.245445 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:20:03.245415    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4724ce74-14df-4bf5-a303-ba98f71b6ebe-tls-cert\") pod \"authorino-5c4697d5b4-9rpkm\" (UID: \"4724ce74-14df-4bf5-a303-ba98f71b6ebe\") " pod="kuadrant-system/authorino-5c4697d5b4-9rpkm"
Jul 07 17:20:03.252566 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:20:03.252535    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bz92n\" (UniqueName: \"kubernetes.io/projected/4724ce74-14df-4bf5-a303-ba98f71b6ebe-kube-api-access-bz92n\") pod \"authorino-5c4697d5b4-9rpkm\" (UID: \"4724ce74-14df-4bf5-a303-ba98f71b6ebe\") " pod="kuadrant-system/authorino-5c4697d5b4-9rpkm"
Jul 07 17:20:03.383457 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:20:03.383396    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5c4697d5b4-9rpkm"
Jul 07 17:20:03.524378 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:20:03.524355    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5c4697d5b4-9rpkm"]
Jul 07 17:20:03.525940 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:20:03.525911    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4724ce74_14df_4bf5_a303_ba98f71b6ebe.slice/crio-348db1d48e1f4eef6358977fa4a2d875f5b651f16d1a934e8eb0853313e980dc WatchSource:0}: Error finding container 348db1d48e1f4eef6358977fa4a2d875f5b651f16d1a934e8eb0853313e980dc: Status 404 returned error can't find the container with id 348db1d48e1f4eef6358977fa4a2d875f5b651f16d1a934e8eb0853313e980dc
Jul 07 17:20:04.515547 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:20:04.515506    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5c4697d5b4-9rpkm" event={"ID":"4724ce74-14df-4bf5-a303-ba98f71b6ebe","Type":"ContainerStarted","Data":"2282b90e429381b74af7c30101d9b8281be29d5034e8c532a406acdfd47f25e3"}
Jul 07 17:20:04.516100 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:20:04.516071    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5c4697d5b4-9rpkm" event={"ID":"4724ce74-14df-4bf5-a303-ba98f71b6ebe","Type":"ContainerStarted","Data":"348db1d48e1f4eef6358977fa4a2d875f5b651f16d1a934e8eb0853313e980dc"}
Jul 07 17:20:04.536968 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:20:04.536918    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5c4697d5b4-9rpkm" podStartSLOduration=0.933318156 podStartE2EDuration="1.536874086s" totalImagesPullingTime="603.55593ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:20:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:20:04.533974698 +0000 UTC m=+728.030053205" watchObservedRunningTime="2026-07-07 17:20:04.536874086 +0000 UTC m=+728.032952593"
Jul 07 17:20:04.571132 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:20:04.571099    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-586b598d45-2pkxd"]
Jul 07 17:20:04.571337 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:20:04.571302    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-586b598d45-2pkxd" podUID="286ceea6-3392-4bf9-889d-dd47a775440a" containerName="authorino" containerID="cri-o://501f5eaddd31c3e255e39c59c61c7cb719b8e4559dc8c2947825d75fe63e318b" gracePeriod=30
Jul 07 17:20:04.837586 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:20:04.837561    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-586b598d45-2pkxd"
Jul 07 17:20:04.855158 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:20:04.855133    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7l5dp\" (UniqueName: \"kubernetes.io/projected/286ceea6-3392-4bf9-889d-dd47a775440a-kube-api-access-7l5dp\") pod \"286ceea6-3392-4bf9-889d-dd47a775440a\" (UID: \"286ceea6-3392-4bf9-889d-dd47a775440a\") "
Jul 07 17:20:04.855319 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:20:04.855208    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/286ceea6-3392-4bf9-889d-dd47a775440a-tls-cert\") pod \"286ceea6-3392-4bf9-889d-dd47a775440a\" (UID: \"286ceea6-3392-4bf9-889d-dd47a775440a\") "
Jul 07 17:20:04.858051 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:20:04.858013    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/286ceea6-3392-4bf9-889d-dd47a775440a-kube-api-access-7l5dp" (OuterVolumeSpecName: "kube-api-access-7l5dp") pod "286ceea6-3392-4bf9-889d-dd47a775440a" (UID: "286ceea6-3392-4bf9-889d-dd47a775440a"). InnerVolumeSpecName "kube-api-access-7l5dp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 17:20:04.869911 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:20:04.869871    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/286ceea6-3392-4bf9-889d-dd47a775440a-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "286ceea6-3392-4bf9-889d-dd47a775440a" (UID: "286ceea6-3392-4bf9-889d-dd47a775440a"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 17:20:04.956590 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:20:04.956559    2579 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/286ceea6-3392-4bf9-889d-dd47a775440a-tls-cert\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:20:04.956590 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:20:04.956585    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7l5dp\" (UniqueName: \"kubernetes.io/projected/286ceea6-3392-4bf9-889d-dd47a775440a-kube-api-access-7l5dp\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:20:05.522311 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:20:05.522234    2579 generic.go:356] "Generic (PLEG): container finished" podID="286ceea6-3392-4bf9-889d-dd47a775440a" containerID="501f5eaddd31c3e255e39c59c61c7cb719b8e4559dc8c2947825d75fe63e318b" exitCode=0
Jul 07 17:20:05.522311 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:20:05.522288    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-586b598d45-2pkxd"
Jul 07 17:20:05.522764 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:20:05.522315    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-586b598d45-2pkxd" event={"ID":"286ceea6-3392-4bf9-889d-dd47a775440a","Type":"ContainerDied","Data":"501f5eaddd31c3e255e39c59c61c7cb719b8e4559dc8c2947825d75fe63e318b"}
Jul 07 17:20:05.522764 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:20:05.522354    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-586b598d45-2pkxd" event={"ID":"286ceea6-3392-4bf9-889d-dd47a775440a","Type":"ContainerDied","Data":"f968cdbdb0a96082541caed48c73cddbd1c9973787182a837c72a4f9503f4379"}
Jul 07 17:20:05.522764 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:20:05.522369    2579 scope.go:117] "RemoveContainer" containerID="501f5eaddd31c3e255e39c59c61c7cb719b8e4559dc8c2947825d75fe63e318b"
Jul 07 17:20:05.535375 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:20:05.535353    2579 scope.go:117] "RemoveContainer" containerID="501f5eaddd31c3e255e39c59c61c7cb719b8e4559dc8c2947825d75fe63e318b"
Jul 07 17:20:05.535765 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:20:05.535728    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"501f5eaddd31c3e255e39c59c61c7cb719b8e4559dc8c2947825d75fe63e318b\": container with ID starting with 501f5eaddd31c3e255e39c59c61c7cb719b8e4559dc8c2947825d75fe63e318b not found: ID does not exist" containerID="501f5eaddd31c3e255e39c59c61c7cb719b8e4559dc8c2947825d75fe63e318b"
Jul 07 17:20:05.535877 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:20:05.535784    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"501f5eaddd31c3e255e39c59c61c7cb719b8e4559dc8c2947825d75fe63e318b"} err="failed to get container status \"501f5eaddd31c3e255e39c59c61c7cb719b8e4559dc8c2947825d75fe63e318b\": rpc error: code = NotFound desc = could not find container \"501f5eaddd31c3e255e39c59c61c7cb719b8e4559dc8c2947825d75fe63e318b\": container with ID starting with 501f5eaddd31c3e255e39c59c61c7cb719b8e4559dc8c2947825d75fe63e318b not found: ID does not exist"
Jul 07 17:20:05.549271 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:20:05.549244    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-586b598d45-2pkxd"]
Jul 07 17:20:05.553322 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:20:05.553295    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-586b598d45-2pkxd"]
Jul 07 17:20:07.027900 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:20:07.027866    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="286ceea6-3392-4bf9-889d-dd47a775440a" path="/var/lib/kubelet/pods/286ceea6-3392-4bf9-889d-dd47a775440a/volumes"
Jul 07 17:37:07.810668 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:07.810636    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-7bdc63cf/e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb"]
Jul 07 17:37:07.811145 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:07.810983    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="286ceea6-3392-4bf9-889d-dd47a775440a" containerName="authorino"
Jul 07 17:37:07.811145 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:07.810996    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="286ceea6-3392-4bf9-889d-dd47a775440a" containerName="authorino"
Jul 07 17:37:07.811145 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:07.811066    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="286ceea6-3392-4bf9-889d-dd47a775440a" containerName="authorino"
Jul 07 17:37:07.813824 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:07.813807    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7bdc63cf/e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb"
Jul 07 17:37:07.817285 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:07.817261    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-7bdc63cf\"/\"default-dockercfg-zfm54\""
Jul 07 17:37:07.817401 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:07.817305    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-7bdc63cf\"/\"e2e-lifecycle-model-7bdc63cf-kserve-self-signed-certs\""
Jul 07 17:37:07.817401 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:07.817337    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-7bdc63cf\"/\"kube-root-ca.crt\""
Jul 07 17:37:07.817484 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:07.817419    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-7bdc63cf\"/\"openshift-service-ca.crt\""
Jul 07 17:37:07.826628 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:07.826608    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-7bdc63cf/e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb"]
Jul 07 17:37:07.862037 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:07.862018    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/156bee6f-2d2e-47ab-87fd-1169c341ac53-dshm\") pod \"e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb\" (UID: \"156bee6f-2d2e-47ab-87fd-1169c341ac53\") " pod="ai-tenant-e2e-mt-7bdc63cf/e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb"
Jul 07 17:37:07.862559 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:07.862528    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/156bee6f-2d2e-47ab-87fd-1169c341ac53-tmp-dir\") pod \"e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb\" (UID: \"156bee6f-2d2e-47ab-87fd-1169c341ac53\") " pod="ai-tenant-e2e-mt-7bdc63cf/e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb"
Jul 07 17:37:07.862650 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:07.862616    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/156bee6f-2d2e-47ab-87fd-1169c341ac53-home\") pod \"e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb\" (UID: \"156bee6f-2d2e-47ab-87fd-1169c341ac53\") " pod="ai-tenant-e2e-mt-7bdc63cf/e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb"
Jul 07 17:37:07.862650 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:07.862639    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/156bee6f-2d2e-47ab-87fd-1169c341ac53-model-cache\") pod \"e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb\" (UID: \"156bee6f-2d2e-47ab-87fd-1169c341ac53\") " pod="ai-tenant-e2e-mt-7bdc63cf/e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb"
Jul 07 17:37:07.862757 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:07.862676    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkfd6\" (UniqueName: \"kubernetes.io/projected/156bee6f-2d2e-47ab-87fd-1169c341ac53-kube-api-access-dkfd6\") pod \"e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb\" (UID: \"156bee6f-2d2e-47ab-87fd-1169c341ac53\") " pod="ai-tenant-e2e-mt-7bdc63cf/e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb"
Jul 07 17:37:07.862810 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:07.862753    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/156bee6f-2d2e-47ab-87fd-1169c341ac53-tls-certs\") pod \"e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb\" (UID: \"156bee6f-2d2e-47ab-87fd-1169c341ac53\") " pod="ai-tenant-e2e-mt-7bdc63cf/e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb"
Jul 07 17:37:07.963292 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:07.963264    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/156bee6f-2d2e-47ab-87fd-1169c341ac53-home\") pod \"e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb\" (UID: \"156bee6f-2d2e-47ab-87fd-1169c341ac53\") " pod="ai-tenant-e2e-mt-7bdc63cf/e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb"
Jul 07 17:37:07.963292 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:07.963292    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/156bee6f-2d2e-47ab-87fd-1169c341ac53-model-cache\") pod \"e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb\" (UID: \"156bee6f-2d2e-47ab-87fd-1169c341ac53\") " pod="ai-tenant-e2e-mt-7bdc63cf/e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb"
Jul 07 17:37:07.963477 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:07.963320    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkfd6\" (UniqueName: \"kubernetes.io/projected/156bee6f-2d2e-47ab-87fd-1169c341ac53-kube-api-access-dkfd6\") pod \"e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb\" (UID: \"156bee6f-2d2e-47ab-87fd-1169c341ac53\") " pod="ai-tenant-e2e-mt-7bdc63cf/e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb"
Jul 07 17:37:07.963477 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:07.963444    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/156bee6f-2d2e-47ab-87fd-1169c341ac53-tls-certs\") pod \"e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb\" (UID: \"156bee6f-2d2e-47ab-87fd-1169c341ac53\") " pod="ai-tenant-e2e-mt-7bdc63cf/e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb"
Jul 07 17:37:07.963598 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:07.963501    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/156bee6f-2d2e-47ab-87fd-1169c341ac53-dshm\") pod \"e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb\" (UID: \"156bee6f-2d2e-47ab-87fd-1169c341ac53\") " pod="ai-tenant-e2e-mt-7bdc63cf/e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb"
Jul 07 17:37:07.963598 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:07.963548    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/156bee6f-2d2e-47ab-87fd-1169c341ac53-tmp-dir\") pod \"e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb\" (UID: \"156bee6f-2d2e-47ab-87fd-1169c341ac53\") " pod="ai-tenant-e2e-mt-7bdc63cf/e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb"
Jul 07 17:37:07.963721 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:07.963702    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/156bee6f-2d2e-47ab-87fd-1169c341ac53-model-cache\") pod \"e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb\" (UID: \"156bee6f-2d2e-47ab-87fd-1169c341ac53\") " pod="ai-tenant-e2e-mt-7bdc63cf/e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb"
Jul 07 17:37:07.963785 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:07.963745    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/156bee6f-2d2e-47ab-87fd-1169c341ac53-home\") pod \"e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb\" (UID: \"156bee6f-2d2e-47ab-87fd-1169c341ac53\") " pod="ai-tenant-e2e-mt-7bdc63cf/e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb"
Jul 07 17:37:07.963923 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:07.963878    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/156bee6f-2d2e-47ab-87fd-1169c341ac53-tmp-dir\") pod \"e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb\" (UID: \"156bee6f-2d2e-47ab-87fd-1169c341ac53\") " pod="ai-tenant-e2e-mt-7bdc63cf/e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb"
Jul 07 17:37:07.965712 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:07.965689    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/156bee6f-2d2e-47ab-87fd-1169c341ac53-dshm\") pod \"e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb\" (UID: \"156bee6f-2d2e-47ab-87fd-1169c341ac53\") " pod="ai-tenant-e2e-mt-7bdc63cf/e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb"
Jul 07 17:37:07.965964 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:07.965949    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/156bee6f-2d2e-47ab-87fd-1169c341ac53-tls-certs\") pod \"e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb\" (UID: \"156bee6f-2d2e-47ab-87fd-1169c341ac53\") " pod="ai-tenant-e2e-mt-7bdc63cf/e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb"
Jul 07 17:37:07.972665 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:07.972630    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkfd6\" (UniqueName: \"kubernetes.io/projected/156bee6f-2d2e-47ab-87fd-1169c341ac53-kube-api-access-dkfd6\") pod \"e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb\" (UID: \"156bee6f-2d2e-47ab-87fd-1169c341ac53\") " pod="ai-tenant-e2e-mt-7bdc63cf/e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb"
Jul 07 17:37:08.122846 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:08.122791    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7bdc63cf/e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb"
Jul 07 17:37:08.251098 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:08.251069    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-7bdc63cf/e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb"]
Jul 07 17:37:08.252538 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:37:08.252507    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod156bee6f_2d2e_47ab_87fd_1169c341ac53.slice/crio-34bd52108b79a6a87d1c181e55af0faa402b456c0d6cac083eaa7bff7f259d84 WatchSource:0}: Error finding container 34bd52108b79a6a87d1c181e55af0faa402b456c0d6cac083eaa7bff7f259d84: Status 404 returned error can't find the container with id 34bd52108b79a6a87d1c181e55af0faa402b456c0d6cac083eaa7bff7f259d84
Jul 07 17:37:08.654907 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:08.654861    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7bdc63cf/e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb" event={"ID":"156bee6f-2d2e-47ab-87fd-1169c341ac53","Type":"ContainerStarted","Data":"cf779619afa3bece3092226aa37de6cf78d1d484dd518b4e1deed506ae8b39e7"}
Jul 07 17:37:08.655078 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:08.654932    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7bdc63cf/e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb" event={"ID":"156bee6f-2d2e-47ab-87fd-1169c341ac53","Type":"ContainerStarted","Data":"34bd52108b79a6a87d1c181e55af0faa402b456c0d6cac083eaa7bff7f259d84"}
Jul 07 17:37:08.674100 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:08.674054    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-7bdc63cf/e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb" podStartSLOduration=1.674039995 podStartE2EDuration="1.674039995s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:37:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:37:08.672054803 +0000 UTC m=+1752.168133310" watchObservedRunningTime="2026-07-07 17:37:08.674039995 +0000 UTC m=+1752.170118551"
Jul 07 17:37:18.123365 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:18.123330    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-7bdc63cf/e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb"
Jul 07 17:37:18.123818 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:18.123379    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-7bdc63cf/e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb"
Jul 07 17:37:18.135455 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:18.135432    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-7bdc63cf/e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb"
Jul 07 17:37:18.717337 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:18.717310    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-7bdc63cf/e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb"
Jul 07 17:37:39.763468 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:39.763431    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-7bdc63cf/e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb"]
Jul 07 17:37:54.783964 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:54.783926    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-7bdc63cf/e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb" podUID="156bee6f-2d2e-47ab-87fd-1169c341ac53" containerName="main" containerID="cri-o://cf779619afa3bece3092226aa37de6cf78d1d484dd518b4e1deed506ae8b39e7" gracePeriod=45
Jul 07 17:37:55.038130 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:55.038074    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7bdc63cf/e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb"
Jul 07 17:37:55.052758 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:55.052741    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/156bee6f-2d2e-47ab-87fd-1169c341ac53-tmp-dir\") pod \"156bee6f-2d2e-47ab-87fd-1169c341ac53\" (UID: \"156bee6f-2d2e-47ab-87fd-1169c341ac53\") "
Jul 07 17:37:55.052820 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:55.052779    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/156bee6f-2d2e-47ab-87fd-1169c341ac53-tls-certs\") pod \"156bee6f-2d2e-47ab-87fd-1169c341ac53\" (UID: \"156bee6f-2d2e-47ab-87fd-1169c341ac53\") "
Jul 07 17:37:55.052820 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:55.052811    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/156bee6f-2d2e-47ab-87fd-1169c341ac53-home\") pod \"156bee6f-2d2e-47ab-87fd-1169c341ac53\" (UID: \"156bee6f-2d2e-47ab-87fd-1169c341ac53\") "
Jul 07 17:37:55.052908 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:55.052872    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dkfd6\" (UniqueName: \"kubernetes.io/projected/156bee6f-2d2e-47ab-87fd-1169c341ac53-kube-api-access-dkfd6\") pod \"156bee6f-2d2e-47ab-87fd-1169c341ac53\" (UID: \"156bee6f-2d2e-47ab-87fd-1169c341ac53\") "
Jul 07 17:37:55.052949 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:55.052932    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/156bee6f-2d2e-47ab-87fd-1169c341ac53-dshm\") pod \"156bee6f-2d2e-47ab-87fd-1169c341ac53\" (UID: \"156bee6f-2d2e-47ab-87fd-1169c341ac53\") "
Jul 07 17:37:55.052983 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:55.052961    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/156bee6f-2d2e-47ab-87fd-1169c341ac53-model-cache\") pod \"156bee6f-2d2e-47ab-87fd-1169c341ac53\" (UID: \"156bee6f-2d2e-47ab-87fd-1169c341ac53\") "
Jul 07 17:37:55.053064 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:55.053027    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/156bee6f-2d2e-47ab-87fd-1169c341ac53-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "156bee6f-2d2e-47ab-87fd-1169c341ac53" (UID: "156bee6f-2d2e-47ab-87fd-1169c341ac53"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:37:55.053126 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:55.053099    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/156bee6f-2d2e-47ab-87fd-1169c341ac53-home" (OuterVolumeSpecName: "home") pod "156bee6f-2d2e-47ab-87fd-1169c341ac53" (UID: "156bee6f-2d2e-47ab-87fd-1169c341ac53"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:37:55.053247 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:55.053226    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/156bee6f-2d2e-47ab-87fd-1169c341ac53-model-cache" (OuterVolumeSpecName: "model-cache") pod "156bee6f-2d2e-47ab-87fd-1169c341ac53" (UID: "156bee6f-2d2e-47ab-87fd-1169c341ac53"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:37:55.053313 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:55.053258    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/156bee6f-2d2e-47ab-87fd-1169c341ac53-tmp-dir\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:37:55.053313 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:55.053274    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/156bee6f-2d2e-47ab-87fd-1169c341ac53-home\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:37:55.055049 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:55.055026    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/156bee6f-2d2e-47ab-87fd-1169c341ac53-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "156bee6f-2d2e-47ab-87fd-1169c341ac53" (UID: "156bee6f-2d2e-47ab-87fd-1169c341ac53"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 17:37:55.055049 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:55.055031    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/156bee6f-2d2e-47ab-87fd-1169c341ac53-dshm" (OuterVolumeSpecName: "dshm") pod "156bee6f-2d2e-47ab-87fd-1169c341ac53" (UID: "156bee6f-2d2e-47ab-87fd-1169c341ac53"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:37:55.055203 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:55.055123    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/156bee6f-2d2e-47ab-87fd-1169c341ac53-kube-api-access-dkfd6" (OuterVolumeSpecName: "kube-api-access-dkfd6") pod "156bee6f-2d2e-47ab-87fd-1169c341ac53" (UID: "156bee6f-2d2e-47ab-87fd-1169c341ac53"). InnerVolumeSpecName "kube-api-access-dkfd6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 17:37:55.153645 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:55.153616    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dkfd6\" (UniqueName: \"kubernetes.io/projected/156bee6f-2d2e-47ab-87fd-1169c341ac53-kube-api-access-dkfd6\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:37:55.153645 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:55.153642    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/156bee6f-2d2e-47ab-87fd-1169c341ac53-dshm\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:37:55.153645 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:55.153651    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/156bee6f-2d2e-47ab-87fd-1169c341ac53-model-cache\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:37:55.153853 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:55.153660    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/156bee6f-2d2e-47ab-87fd-1169c341ac53-tls-certs\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:37:55.898375 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:55.898339    2579 generic.go:356] "Generic (PLEG): container finished" podID="156bee6f-2d2e-47ab-87fd-1169c341ac53" containerID="cf779619afa3bece3092226aa37de6cf78d1d484dd518b4e1deed506ae8b39e7" exitCode=0
Jul 07 17:37:55.898810 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:55.898389    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7bdc63cf/e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb"
Jul 07 17:37:55.898810 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:55.898427    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7bdc63cf/e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb" event={"ID":"156bee6f-2d2e-47ab-87fd-1169c341ac53","Type":"ContainerDied","Data":"cf779619afa3bece3092226aa37de6cf78d1d484dd518b4e1deed506ae8b39e7"}
Jul 07 17:37:55.898810 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:55.898475    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7bdc63cf/e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb" event={"ID":"156bee6f-2d2e-47ab-87fd-1169c341ac53","Type":"ContainerDied","Data":"34bd52108b79a6a87d1c181e55af0faa402b456c0d6cac083eaa7bff7f259d84"}
Jul 07 17:37:55.898810 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:55.898497    2579 scope.go:117] "RemoveContainer" containerID="cf779619afa3bece3092226aa37de6cf78d1d484dd518b4e1deed506ae8b39e7"
Jul 07 17:37:55.907853 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:55.907837    2579 scope.go:117] "RemoveContainer" containerID="cf779619afa3bece3092226aa37de6cf78d1d484dd518b4e1deed506ae8b39e7"
Jul 07 17:37:55.908121 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:37:55.908103    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf779619afa3bece3092226aa37de6cf78d1d484dd518b4e1deed506ae8b39e7\": container with ID starting with cf779619afa3bece3092226aa37de6cf78d1d484dd518b4e1deed506ae8b39e7 not found: ID does not exist" containerID="cf779619afa3bece3092226aa37de6cf78d1d484dd518b4e1deed506ae8b39e7"
Jul 07 17:37:55.908227 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:55.908127    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf779619afa3bece3092226aa37de6cf78d1d484dd518b4e1deed506ae8b39e7"} err="failed to get container status \"cf779619afa3bece3092226aa37de6cf78d1d484dd518b4e1deed506ae8b39e7\": rpc error: code = NotFound desc = could not find container \"cf779619afa3bece3092226aa37de6cf78d1d484dd518b4e1deed506ae8b39e7\": container with ID starting with cf779619afa3bece3092226aa37de6cf78d1d484dd518b4e1deed506ae8b39e7 not found: ID does not exist"
Jul 07 17:37:55.920499 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:55.920473    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-7bdc63cf/e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb"]
Jul 07 17:37:55.925203 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:55.925185    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-7bdc63cf/e2e-lifecycle-model-7bdc63cf-kserve-58d99458f9-qbnqb"]
Jul 07 17:37:57.028203 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:37:57.028160    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="156bee6f-2d2e-47ab-87fd-1169c341ac53" path="/var/lib/kubelet/pods/156bee6f-2d2e-47ab-87fd-1169c341ac53/volumes"
Jul 07 17:39:53.317408 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:39:53.317374    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-457985/test-model-457985-kserve-74895fbf97-qrnc9"]
Jul 07 17:39:53.319611 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:39:53.317698    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="156bee6f-2d2e-47ab-87fd-1169c341ac53" containerName="main"
Jul 07 17:39:53.319611 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:39:53.317709    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="156bee6f-2d2e-47ab-87fd-1169c341ac53" containerName="main"
Jul 07 17:39:53.319611 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:39:53.317772    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="156bee6f-2d2e-47ab-87fd-1169c341ac53" containerName="main"
Jul 07 17:39:53.320438 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:39:53.320423    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-457985/test-model-457985-kserve-74895fbf97-qrnc9"
Jul 07 17:39:53.322824 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:39:53.322795    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-457985\"/\"openshift-service-ca.crt\""
Jul 07 17:39:53.322967 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:39:53.322828    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-457985\"/\"kube-root-ca.crt\""
Jul 07 17:39:53.323670 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:39:53.323653    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-457985\"/\"default-dockercfg-cfqd8\""
Jul 07 17:39:53.323724 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:39:53.323700    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-457985\"/\"test-model-457985-kserve-self-signed-certs\""
Jul 07 17:39:53.336465 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:39:53.336445    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-457985/test-model-457985-kserve-74895fbf97-qrnc9"]
Jul 07 17:39:53.459277 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:39:53.459246    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf32513a-2abf-402f-974b-6c75280d21a2-model-cache\") pod \"test-model-457985-kserve-74895fbf97-qrnc9\" (UID: \"bf32513a-2abf-402f-974b-6c75280d21a2\") " pod="ai-tenant-e2e-inf-a-457985/test-model-457985-kserve-74895fbf97-qrnc9"
Jul 07 17:39:53.459277 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:39:53.459281    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf32513a-2abf-402f-974b-6c75280d21a2-dshm\") pod \"test-model-457985-kserve-74895fbf97-qrnc9\" (UID: \"bf32513a-2abf-402f-974b-6c75280d21a2\") " pod="ai-tenant-e2e-inf-a-457985/test-model-457985-kserve-74895fbf97-qrnc9"
Jul 07 17:39:53.459477 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:39:53.459342    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf32513a-2abf-402f-974b-6c75280d21a2-tls-certs\") pod \"test-model-457985-kserve-74895fbf97-qrnc9\" (UID: \"bf32513a-2abf-402f-974b-6c75280d21a2\") " pod="ai-tenant-e2e-inf-a-457985/test-model-457985-kserve-74895fbf97-qrnc9"
Jul 07 17:39:53.459477 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:39:53.459372    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqbbn\" (UniqueName: \"kubernetes.io/projected/bf32513a-2abf-402f-974b-6c75280d21a2-kube-api-access-kqbbn\") pod \"test-model-457985-kserve-74895fbf97-qrnc9\" (UID: \"bf32513a-2abf-402f-974b-6c75280d21a2\") " pod="ai-tenant-e2e-inf-a-457985/test-model-457985-kserve-74895fbf97-qrnc9"
Jul 07 17:39:53.459477 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:39:53.459405    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf32513a-2abf-402f-974b-6c75280d21a2-home\") pod \"test-model-457985-kserve-74895fbf97-qrnc9\" (UID: \"bf32513a-2abf-402f-974b-6c75280d21a2\") " pod="ai-tenant-e2e-inf-a-457985/test-model-457985-kserve-74895fbf97-qrnc9"
Jul 07 17:39:53.459477 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:39:53.459452    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf32513a-2abf-402f-974b-6c75280d21a2-tmp-dir\") pod \"test-model-457985-kserve-74895fbf97-qrnc9\" (UID: \"bf32513a-2abf-402f-974b-6c75280d21a2\") " pod="ai-tenant-e2e-inf-a-457985/test-model-457985-kserve-74895fbf97-qrnc9"
Jul 07 17:39:53.560389 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:39:53.560354    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf32513a-2abf-402f-974b-6c75280d21a2-tmp-dir\") pod \"test-model-457985-kserve-74895fbf97-qrnc9\" (UID: \"bf32513a-2abf-402f-974b-6c75280d21a2\") " pod="ai-tenant-e2e-inf-a-457985/test-model-457985-kserve-74895fbf97-qrnc9"
Jul 07 17:39:53.560570 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:39:53.560398    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf32513a-2abf-402f-974b-6c75280d21a2-model-cache\") pod \"test-model-457985-kserve-74895fbf97-qrnc9\" (UID: \"bf32513a-2abf-402f-974b-6c75280d21a2\") " pod="ai-tenant-e2e-inf-a-457985/test-model-457985-kserve-74895fbf97-qrnc9"
Jul 07 17:39:53.560570 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:39:53.560426    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf32513a-2abf-402f-974b-6c75280d21a2-dshm\") pod \"test-model-457985-kserve-74895fbf97-qrnc9\" (UID: \"bf32513a-2abf-402f-974b-6c75280d21a2\") " pod="ai-tenant-e2e-inf-a-457985/test-model-457985-kserve-74895fbf97-qrnc9"
Jul 07 17:39:53.560570 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:39:53.560484    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf32513a-2abf-402f-974b-6c75280d21a2-tls-certs\") pod \"test-model-457985-kserve-74895fbf97-qrnc9\" (UID: \"bf32513a-2abf-402f-974b-6c75280d21a2\") " pod="ai-tenant-e2e-inf-a-457985/test-model-457985-kserve-74895fbf97-qrnc9"
Jul 07 17:39:53.560570 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:39:53.560509    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqbbn\" (UniqueName: \"kubernetes.io/projected/bf32513a-2abf-402f-974b-6c75280d21a2-kube-api-access-kqbbn\") pod \"test-model-457985-kserve-74895fbf97-qrnc9\" (UID: \"bf32513a-2abf-402f-974b-6c75280d21a2\") " pod="ai-tenant-e2e-inf-a-457985/test-model-457985-kserve-74895fbf97-qrnc9"
Jul 07 17:39:53.560570 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:39:53.560549    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf32513a-2abf-402f-974b-6c75280d21a2-home\") pod \"test-model-457985-kserve-74895fbf97-qrnc9\" (UID: \"bf32513a-2abf-402f-974b-6c75280d21a2\") " pod="ai-tenant-e2e-inf-a-457985/test-model-457985-kserve-74895fbf97-qrnc9"
Jul 07 17:39:53.560824 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:39:53.560807    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf32513a-2abf-402f-974b-6c75280d21a2-model-cache\") pod \"test-model-457985-kserve-74895fbf97-qrnc9\" (UID: \"bf32513a-2abf-402f-974b-6c75280d21a2\") " pod="ai-tenant-e2e-inf-a-457985/test-model-457985-kserve-74895fbf97-qrnc9"
Jul 07 17:39:53.560824 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:39:53.560817    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf32513a-2abf-402f-974b-6c75280d21a2-tmp-dir\") pod \"test-model-457985-kserve-74895fbf97-qrnc9\" (UID: \"bf32513a-2abf-402f-974b-6c75280d21a2\") " pod="ai-tenant-e2e-inf-a-457985/test-model-457985-kserve-74895fbf97-qrnc9"
Jul 07 17:39:53.560981 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:39:53.560962    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf32513a-2abf-402f-974b-6c75280d21a2-home\") pod \"test-model-457985-kserve-74895fbf97-qrnc9\" (UID: \"bf32513a-2abf-402f-974b-6c75280d21a2\") " pod="ai-tenant-e2e-inf-a-457985/test-model-457985-kserve-74895fbf97-qrnc9"
Jul 07 17:39:53.562903 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:39:53.562867    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf32513a-2abf-402f-974b-6c75280d21a2-dshm\") pod \"test-model-457985-kserve-74895fbf97-qrnc9\" (UID: \"bf32513a-2abf-402f-974b-6c75280d21a2\") " pod="ai-tenant-e2e-inf-a-457985/test-model-457985-kserve-74895fbf97-qrnc9"
Jul 07 17:39:53.563212 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:39:53.563195    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf32513a-2abf-402f-974b-6c75280d21a2-tls-certs\") pod \"test-model-457985-kserve-74895fbf97-qrnc9\" (UID: \"bf32513a-2abf-402f-974b-6c75280d21a2\") " pod="ai-tenant-e2e-inf-a-457985/test-model-457985-kserve-74895fbf97-qrnc9"
Jul 07 17:39:53.569559 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:39:53.569510    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqbbn\" (UniqueName: \"kubernetes.io/projected/bf32513a-2abf-402f-974b-6c75280d21a2-kube-api-access-kqbbn\") pod \"test-model-457985-kserve-74895fbf97-qrnc9\" (UID: \"bf32513a-2abf-402f-974b-6c75280d21a2\") " pod="ai-tenant-e2e-inf-a-457985/test-model-457985-kserve-74895fbf97-qrnc9"
Jul 07 17:39:53.629823 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:39:53.629797    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-457985/test-model-457985-kserve-74895fbf97-qrnc9"
Jul 07 17:39:53.763951 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:39:53.763923    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-457985/test-model-457985-kserve-74895fbf97-qrnc9"]
Jul 07 17:39:53.765826 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:39:53.765800    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbf32513a_2abf_402f_974b_6c75280d21a2.slice/crio-b53a44401ee2b94f26e0b03d3987fe99ef0917ae972affe519442d2d5783fb81 WatchSource:0}: Error finding container b53a44401ee2b94f26e0b03d3987fe99ef0917ae972affe519442d2d5783fb81: Status 404 returned error can't find the container with id b53a44401ee2b94f26e0b03d3987fe99ef0917ae972affe519442d2d5783fb81
Jul 07 17:39:54.511081 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:39:54.511037    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-457985/test-model-457985-kserve-74895fbf97-qrnc9" event={"ID":"bf32513a-2abf-402f-974b-6c75280d21a2","Type":"ContainerStarted","Data":"92e9baeec0e8b186423c2ba4fab8c93944ffe41e0c324b5d8c9815fc0825b7af"}
Jul 07 17:39:54.511081 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:39:54.511085    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-457985/test-model-457985-kserve-74895fbf97-qrnc9" event={"ID":"bf32513a-2abf-402f-974b-6c75280d21a2","Type":"ContainerStarted","Data":"b53a44401ee2b94f26e0b03d3987fe99ef0917ae972affe519442d2d5783fb81"}
Jul 07 17:39:54.530189 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:39:54.530143    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-457985/test-model-457985-kserve-74895fbf97-qrnc9" podStartSLOduration=1.530130293 podStartE2EDuration="1.530130293s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:39:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:39:54.529244047 +0000 UTC m=+1918.025322555" watchObservedRunningTime="2026-07-07 17:39:54.530130293 +0000 UTC m=+1918.026209183"
Jul 07 17:40:03.630871 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:03.630826    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-457985/test-model-457985-kserve-74895fbf97-qrnc9"
Jul 07 17:40:03.631310 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:03.630882    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-457985/test-model-457985-kserve-74895fbf97-qrnc9"
Jul 07 17:40:03.643697 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:03.643669    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-457985/test-model-457985-kserve-74895fbf97-qrnc9"
Jul 07 17:40:04.573261 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:04.573231    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-457985/test-model-457985-kserve-74895fbf97-qrnc9"
Jul 07 17:40:11.178840 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:11.178795    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-567860/test-model-567860-kserve-59777bcd4f-xxl7c"]
Jul 07 17:40:11.182381 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:11.182354    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-567860/test-model-567860-kserve-59777bcd4f-xxl7c"
Jul 07 17:40:11.184720 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:11.184694    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-567860\"/\"kube-root-ca.crt\""
Jul 07 17:40:11.185620 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:11.185598    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-567860\"/\"test-model-567860-kserve-self-signed-certs\""
Jul 07 17:40:11.185752 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:11.185633    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-567860\"/\"default-dockercfg-stth4\""
Jul 07 17:40:11.185752 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:11.185642    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-567860\"/\"openshift-service-ca.crt\""
Jul 07 17:40:11.199922 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:11.199874    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-567860/test-model-567860-kserve-59777bcd4f-xxl7c"]
Jul 07 17:40:11.324134 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:11.324088    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49d9c976-0322-431a-88af-4c2dda8895fc-model-cache\") pod \"test-model-567860-kserve-59777bcd4f-xxl7c\" (UID: \"49d9c976-0322-431a-88af-4c2dda8895fc\") " pod="ai-tenant-e2e-inf-b-567860/test-model-567860-kserve-59777bcd4f-xxl7c"
Jul 07 17:40:11.324354 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:11.324169    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49d9c976-0322-431a-88af-4c2dda8895fc-home\") pod \"test-model-567860-kserve-59777bcd4f-xxl7c\" (UID: \"49d9c976-0322-431a-88af-4c2dda8895fc\") " pod="ai-tenant-e2e-inf-b-567860/test-model-567860-kserve-59777bcd4f-xxl7c"
Jul 07 17:40:11.324354 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:11.324194    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49d9c976-0322-431a-88af-4c2dda8895fc-tmp-dir\") pod \"test-model-567860-kserve-59777bcd4f-xxl7c\" (UID: \"49d9c976-0322-431a-88af-4c2dda8895fc\") " pod="ai-tenant-e2e-inf-b-567860/test-model-567860-kserve-59777bcd4f-xxl7c"
Jul 07 17:40:11.324354 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:11.324271    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85cmp\" (UniqueName: \"kubernetes.io/projected/49d9c976-0322-431a-88af-4c2dda8895fc-kube-api-access-85cmp\") pod \"test-model-567860-kserve-59777bcd4f-xxl7c\" (UID: \"49d9c976-0322-431a-88af-4c2dda8895fc\") " pod="ai-tenant-e2e-inf-b-567860/test-model-567860-kserve-59777bcd4f-xxl7c"
Jul 07 17:40:11.324354 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:11.324306    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49d9c976-0322-431a-88af-4c2dda8895fc-dshm\") pod \"test-model-567860-kserve-59777bcd4f-xxl7c\" (UID: \"49d9c976-0322-431a-88af-4c2dda8895fc\") " pod="ai-tenant-e2e-inf-b-567860/test-model-567860-kserve-59777bcd4f-xxl7c"
Jul 07 17:40:11.324354 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:11.324335    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49d9c976-0322-431a-88af-4c2dda8895fc-tls-certs\") pod \"test-model-567860-kserve-59777bcd4f-xxl7c\" (UID: \"49d9c976-0322-431a-88af-4c2dda8895fc\") " pod="ai-tenant-e2e-inf-b-567860/test-model-567860-kserve-59777bcd4f-xxl7c"
Jul 07 17:40:11.425502 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:11.425455    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49d9c976-0322-431a-88af-4c2dda8895fc-dshm\") pod \"test-model-567860-kserve-59777bcd4f-xxl7c\" (UID: \"49d9c976-0322-431a-88af-4c2dda8895fc\") " pod="ai-tenant-e2e-inf-b-567860/test-model-567860-kserve-59777bcd4f-xxl7c"
Jul 07 17:40:11.425502 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:11.425506    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49d9c976-0322-431a-88af-4c2dda8895fc-tls-certs\") pod \"test-model-567860-kserve-59777bcd4f-xxl7c\" (UID: \"49d9c976-0322-431a-88af-4c2dda8895fc\") " pod="ai-tenant-e2e-inf-b-567860/test-model-567860-kserve-59777bcd4f-xxl7c"
Jul 07 17:40:11.425771 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:11.425542    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49d9c976-0322-431a-88af-4c2dda8895fc-model-cache\") pod \"test-model-567860-kserve-59777bcd4f-xxl7c\" (UID: \"49d9c976-0322-431a-88af-4c2dda8895fc\") " pod="ai-tenant-e2e-inf-b-567860/test-model-567860-kserve-59777bcd4f-xxl7c"
Jul 07 17:40:11.425771 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:11.425620    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49d9c976-0322-431a-88af-4c2dda8895fc-home\") pod \"test-model-567860-kserve-59777bcd4f-xxl7c\" (UID: \"49d9c976-0322-431a-88af-4c2dda8895fc\") " pod="ai-tenant-e2e-inf-b-567860/test-model-567860-kserve-59777bcd4f-xxl7c"
Jul 07 17:40:11.425771 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:11.425642    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49d9c976-0322-431a-88af-4c2dda8895fc-tmp-dir\") pod \"test-model-567860-kserve-59777bcd4f-xxl7c\" (UID: \"49d9c976-0322-431a-88af-4c2dda8895fc\") " pod="ai-tenant-e2e-inf-b-567860/test-model-567860-kserve-59777bcd4f-xxl7c"
Jul 07 17:40:11.425771 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:11.425737    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85cmp\" (UniqueName: \"kubernetes.io/projected/49d9c976-0322-431a-88af-4c2dda8895fc-kube-api-access-85cmp\") pod \"test-model-567860-kserve-59777bcd4f-xxl7c\" (UID: \"49d9c976-0322-431a-88af-4c2dda8895fc\") " pod="ai-tenant-e2e-inf-b-567860/test-model-567860-kserve-59777bcd4f-xxl7c"
Jul 07 17:40:11.426024 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:11.425999    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49d9c976-0322-431a-88af-4c2dda8895fc-tmp-dir\") pod \"test-model-567860-kserve-59777bcd4f-xxl7c\" (UID: \"49d9c976-0322-431a-88af-4c2dda8895fc\") " pod="ai-tenant-e2e-inf-b-567860/test-model-567860-kserve-59777bcd4f-xxl7c"
Jul 07 17:40:11.426081 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:11.426035    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49d9c976-0322-431a-88af-4c2dda8895fc-model-cache\") pod \"test-model-567860-kserve-59777bcd4f-xxl7c\" (UID: \"49d9c976-0322-431a-88af-4c2dda8895fc\") " pod="ai-tenant-e2e-inf-b-567860/test-model-567860-kserve-59777bcd4f-xxl7c"
Jul 07 17:40:11.426081 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:11.426054    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49d9c976-0322-431a-88af-4c2dda8895fc-home\") pod \"test-model-567860-kserve-59777bcd4f-xxl7c\" (UID: \"49d9c976-0322-431a-88af-4c2dda8895fc\") " pod="ai-tenant-e2e-inf-b-567860/test-model-567860-kserve-59777bcd4f-xxl7c"
Jul 07 17:40:11.428028 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:11.428006    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49d9c976-0322-431a-88af-4c2dda8895fc-dshm\") pod \"test-model-567860-kserve-59777bcd4f-xxl7c\" (UID: \"49d9c976-0322-431a-88af-4c2dda8895fc\") " pod="ai-tenant-e2e-inf-b-567860/test-model-567860-kserve-59777bcd4f-xxl7c"
Jul 07 17:40:11.428402 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:11.428381    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49d9c976-0322-431a-88af-4c2dda8895fc-tls-certs\") pod \"test-model-567860-kserve-59777bcd4f-xxl7c\" (UID: \"49d9c976-0322-431a-88af-4c2dda8895fc\") " pod="ai-tenant-e2e-inf-b-567860/test-model-567860-kserve-59777bcd4f-xxl7c"
Jul 07 17:40:11.435144 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:11.435077    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-85cmp\" (UniqueName: \"kubernetes.io/projected/49d9c976-0322-431a-88af-4c2dda8895fc-kube-api-access-85cmp\") pod \"test-model-567860-kserve-59777bcd4f-xxl7c\" (UID: \"49d9c976-0322-431a-88af-4c2dda8895fc\") " pod="ai-tenant-e2e-inf-b-567860/test-model-567860-kserve-59777bcd4f-xxl7c"
Jul 07 17:40:11.491972 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:11.491924    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-567860/test-model-567860-kserve-59777bcd4f-xxl7c"
Jul 07 17:40:11.637029 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:11.636997    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-567860/test-model-567860-kserve-59777bcd4f-xxl7c"]
Jul 07 17:40:11.638904 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:40:11.638862    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod49d9c976_0322_431a_88af_4c2dda8895fc.slice/crio-16487391657f098c0c53e9ba7f35ecce3af11b383b9c8ad486d4039de763b79f WatchSource:0}: Error finding container 16487391657f098c0c53e9ba7f35ecce3af11b383b9c8ad486d4039de763b79f: Status 404 returned error can't find the container with id 16487391657f098c0c53e9ba7f35ecce3af11b383b9c8ad486d4039de763b79f
Jul 07 17:40:12.606465 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:12.606425    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-567860/test-model-567860-kserve-59777bcd4f-xxl7c" event={"ID":"49d9c976-0322-431a-88af-4c2dda8895fc","Type":"ContainerStarted","Data":"80c95f3e000fb98357e45f9465412d727cd3d763091eadbaa0b8829c8d78663a"}
Jul 07 17:40:12.606465 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:12.606471    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-567860/test-model-567860-kserve-59777bcd4f-xxl7c" event={"ID":"49d9c976-0322-431a-88af-4c2dda8895fc","Type":"ContainerStarted","Data":"16487391657f098c0c53e9ba7f35ecce3af11b383b9c8ad486d4039de763b79f"}
Jul 07 17:40:12.626951 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:12.626870    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-567860/test-model-567860-kserve-59777bcd4f-xxl7c" podStartSLOduration=1.626849424 podStartE2EDuration="1.626849424s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:40:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:40:12.625140779 +0000 UTC m=+1936.121219309" watchObservedRunningTime="2026-07-07 17:40:12.626849424 +0000 UTC m=+1936.122927933"
Jul 07 17:40:21.492286 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:21.492243    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-567860/test-model-567860-kserve-59777bcd4f-xxl7c"
Jul 07 17:40:21.492734 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:21.492310    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-567860/test-model-567860-kserve-59777bcd4f-xxl7c"
Jul 07 17:40:21.504997 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:21.504960    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-567860/test-model-567860-kserve-59777bcd4f-xxl7c"
Jul 07 17:40:21.666422 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:21.666387    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-567860/test-model-567860-kserve-59777bcd4f-xxl7c"
Jul 07 17:40:30.325833 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:30.325791    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-457985/test-model-457985-kserve-74895fbf97-qrnc9"]
Jul 07 17:40:31.216923 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:31.215117    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-567860/test-model-567860-kserve-59777bcd4f-xxl7c"]
Jul 07 17:40:45.345820 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:45.345759    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-457985/test-model-457985-kserve-74895fbf97-qrnc9" podUID="bf32513a-2abf-402f-974b-6c75280d21a2" containerName="main" containerID="cri-o://92e9baeec0e8b186423c2ba4fab8c93944ffe41e0c324b5d8c9815fc0825b7af" gracePeriod=45
Jul 07 17:40:45.606734 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:45.606663    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-457985/test-model-457985-kserve-74895fbf97-qrnc9"
Jul 07 17:40:45.724940 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:45.724867    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf32513a-2abf-402f-974b-6c75280d21a2-tls-certs\") pod \"bf32513a-2abf-402f-974b-6c75280d21a2\" (UID: \"bf32513a-2abf-402f-974b-6c75280d21a2\") "
Jul 07 17:40:45.725141 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:45.725002    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kqbbn\" (UniqueName: \"kubernetes.io/projected/bf32513a-2abf-402f-974b-6c75280d21a2-kube-api-access-kqbbn\") pod \"bf32513a-2abf-402f-974b-6c75280d21a2\" (UID: \"bf32513a-2abf-402f-974b-6c75280d21a2\") "
Jul 07 17:40:45.725141 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:45.725032    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf32513a-2abf-402f-974b-6c75280d21a2-dshm\") pod \"bf32513a-2abf-402f-974b-6c75280d21a2\" (UID: \"bf32513a-2abf-402f-974b-6c75280d21a2\") "
Jul 07 17:40:45.725141 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:45.725050    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf32513a-2abf-402f-974b-6c75280d21a2-tmp-dir\") pod \"bf32513a-2abf-402f-974b-6c75280d21a2\" (UID: \"bf32513a-2abf-402f-974b-6c75280d21a2\") "
Jul 07 17:40:45.725141 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:45.725070    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf32513a-2abf-402f-974b-6c75280d21a2-model-cache\") pod \"bf32513a-2abf-402f-974b-6c75280d21a2\" (UID: \"bf32513a-2abf-402f-974b-6c75280d21a2\") "
Jul 07 17:40:45.725141 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:45.725089    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf32513a-2abf-402f-974b-6c75280d21a2-home\") pod \"bf32513a-2abf-402f-974b-6c75280d21a2\" (UID: \"bf32513a-2abf-402f-974b-6c75280d21a2\") "
Jul 07 17:40:45.725411 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:45.725337    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf32513a-2abf-402f-974b-6c75280d21a2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bf32513a-2abf-402f-974b-6c75280d21a2" (UID: "bf32513a-2abf-402f-974b-6c75280d21a2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:40:45.725478 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:45.725419    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf32513a-2abf-402f-974b-6c75280d21a2-model-cache" (OuterVolumeSpecName: "model-cache") pod "bf32513a-2abf-402f-974b-6c75280d21a2" (UID: "bf32513a-2abf-402f-974b-6c75280d21a2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:40:45.725478 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:45.725441    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf32513a-2abf-402f-974b-6c75280d21a2-home" (OuterVolumeSpecName: "home") pod "bf32513a-2abf-402f-974b-6c75280d21a2" (UID: "bf32513a-2abf-402f-974b-6c75280d21a2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:40:45.727460 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:45.727432    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf32513a-2abf-402f-974b-6c75280d21a2-dshm" (OuterVolumeSpecName: "dshm") pod "bf32513a-2abf-402f-974b-6c75280d21a2" (UID: "bf32513a-2abf-402f-974b-6c75280d21a2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:40:45.727567 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:45.727497    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf32513a-2abf-402f-974b-6c75280d21a2-kube-api-access-kqbbn" (OuterVolumeSpecName: "kube-api-access-kqbbn") pod "bf32513a-2abf-402f-974b-6c75280d21a2" (UID: "bf32513a-2abf-402f-974b-6c75280d21a2"). InnerVolumeSpecName "kube-api-access-kqbbn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 17:40:45.727567 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:45.727503    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf32513a-2abf-402f-974b-6c75280d21a2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bf32513a-2abf-402f-974b-6c75280d21a2" (UID: "bf32513a-2abf-402f-974b-6c75280d21a2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 17:40:45.782123 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:45.782087    2579 generic.go:356] "Generic (PLEG): container finished" podID="bf32513a-2abf-402f-974b-6c75280d21a2" containerID="92e9baeec0e8b186423c2ba4fab8c93944ffe41e0c324b5d8c9815fc0825b7af" exitCode=0
Jul 07 17:40:45.782342 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:45.782131    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-457985/test-model-457985-kserve-74895fbf97-qrnc9"
Jul 07 17:40:45.782342 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:45.782131    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-457985/test-model-457985-kserve-74895fbf97-qrnc9" event={"ID":"bf32513a-2abf-402f-974b-6c75280d21a2","Type":"ContainerDied","Data":"92e9baeec0e8b186423c2ba4fab8c93944ffe41e0c324b5d8c9815fc0825b7af"}
Jul 07 17:40:45.782342 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:45.782189    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-457985/test-model-457985-kserve-74895fbf97-qrnc9" event={"ID":"bf32513a-2abf-402f-974b-6c75280d21a2","Type":"ContainerDied","Data":"b53a44401ee2b94f26e0b03d3987fe99ef0917ae972affe519442d2d5783fb81"}
Jul 07 17:40:45.782342 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:45.782213    2579 scope.go:117] "RemoveContainer" containerID="92e9baeec0e8b186423c2ba4fab8c93944ffe41e0c324b5d8c9815fc0825b7af"
Jul 07 17:40:45.792816 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:45.792797    2579 scope.go:117] "RemoveContainer" containerID="92e9baeec0e8b186423c2ba4fab8c93944ffe41e0c324b5d8c9815fc0825b7af"
Jul 07 17:40:45.793118 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:40:45.793100    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92e9baeec0e8b186423c2ba4fab8c93944ffe41e0c324b5d8c9815fc0825b7af\": container with ID starting with 92e9baeec0e8b186423c2ba4fab8c93944ffe41e0c324b5d8c9815fc0825b7af not found: ID does not exist" containerID="92e9baeec0e8b186423c2ba4fab8c93944ffe41e0c324b5d8c9815fc0825b7af"
Jul 07 17:40:45.793177 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:45.793126    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92e9baeec0e8b186423c2ba4fab8c93944ffe41e0c324b5d8c9815fc0825b7af"} err="failed to get container status \"92e9baeec0e8b186423c2ba4fab8c93944ffe41e0c324b5d8c9815fc0825b7af\": rpc error: code = NotFound desc = could not find container \"92e9baeec0e8b186423c2ba4fab8c93944ffe41e0c324b5d8c9815fc0825b7af\": container with ID starting with 92e9baeec0e8b186423c2ba4fab8c93944ffe41e0c324b5d8c9815fc0825b7af not found: ID does not exist"
Jul 07 17:40:45.806422 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:45.806387    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-457985/test-model-457985-kserve-74895fbf97-qrnc9"]
Jul 07 17:40:45.811744 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:45.811710    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-457985/test-model-457985-kserve-74895fbf97-qrnc9"]
Jul 07 17:40:45.826225 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:45.826184    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kqbbn\" (UniqueName: \"kubernetes.io/projected/bf32513a-2abf-402f-974b-6c75280d21a2-kube-api-access-kqbbn\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:40:45.826225 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:45.826219    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf32513a-2abf-402f-974b-6c75280d21a2-dshm\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:40:45.826225 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:45.826230    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf32513a-2abf-402f-974b-6c75280d21a2-tmp-dir\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:40:45.826458 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:45.826241    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf32513a-2abf-402f-974b-6c75280d21a2-model-cache\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:40:45.826458 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:45.826250    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf32513a-2abf-402f-974b-6c75280d21a2-home\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:40:45.826458 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:45.826257    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf32513a-2abf-402f-974b-6c75280d21a2-tls-certs\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:40:46.238391 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:46.238350    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-567860/test-model-567860-kserve-59777bcd4f-xxl7c" podUID="49d9c976-0322-431a-88af-4c2dda8895fc" containerName="main" containerID="cri-o://80c95f3e000fb98357e45f9465412d727cd3d763091eadbaa0b8829c8d78663a" gracePeriod=45
Jul 07 17:40:46.493320 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:46.493251    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-567860/test-model-567860-kserve-59777bcd4f-xxl7c"
Jul 07 17:40:46.632808 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:46.632765    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49d9c976-0322-431a-88af-4c2dda8895fc-dshm\") pod \"49d9c976-0322-431a-88af-4c2dda8895fc\" (UID: \"49d9c976-0322-431a-88af-4c2dda8895fc\") "
Jul 07 17:40:46.633039 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:46.632857    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49d9c976-0322-431a-88af-4c2dda8895fc-model-cache\") pod \"49d9c976-0322-431a-88af-4c2dda8895fc\" (UID: \"49d9c976-0322-431a-88af-4c2dda8895fc\") "
Jul 07 17:40:46.633039 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:46.632877    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49d9c976-0322-431a-88af-4c2dda8895fc-tmp-dir\") pod \"49d9c976-0322-431a-88af-4c2dda8895fc\" (UID: \"49d9c976-0322-431a-88af-4c2dda8895fc\") "
Jul 07 17:40:46.633039 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:46.632935    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-85cmp\" (UniqueName: \"kubernetes.io/projected/49d9c976-0322-431a-88af-4c2dda8895fc-kube-api-access-85cmp\") pod \"49d9c976-0322-431a-88af-4c2dda8895fc\" (UID: \"49d9c976-0322-431a-88af-4c2dda8895fc\") "
Jul 07 17:40:46.633039 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:46.632955    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49d9c976-0322-431a-88af-4c2dda8895fc-home\") pod \"49d9c976-0322-431a-88af-4c2dda8895fc\" (UID: \"49d9c976-0322-431a-88af-4c2dda8895fc\") "
Jul 07 17:40:46.633039 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:46.632974    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49d9c976-0322-431a-88af-4c2dda8895fc-tls-certs\") pod \"49d9c976-0322-431a-88af-4c2dda8895fc\" (UID: \"49d9c976-0322-431a-88af-4c2dda8895fc\") "
Jul 07 17:40:46.633294 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:46.633191    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49d9c976-0322-431a-88af-4c2dda8895fc-model-cache" (OuterVolumeSpecName: "model-cache") pod "49d9c976-0322-431a-88af-4c2dda8895fc" (UID: "49d9c976-0322-431a-88af-4c2dda8895fc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:40:46.633294 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:46.633207    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49d9c976-0322-431a-88af-4c2dda8895fc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "49d9c976-0322-431a-88af-4c2dda8895fc" (UID: "49d9c976-0322-431a-88af-4c2dda8895fc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:40:46.633368 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:46.633307    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49d9c976-0322-431a-88af-4c2dda8895fc-home" (OuterVolumeSpecName: "home") pod "49d9c976-0322-431a-88af-4c2dda8895fc" (UID: "49d9c976-0322-431a-88af-4c2dda8895fc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:40:46.635386 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:46.635355    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49d9c976-0322-431a-88af-4c2dda8895fc-kube-api-access-85cmp" (OuterVolumeSpecName: "kube-api-access-85cmp") pod "49d9c976-0322-431a-88af-4c2dda8895fc" (UID: "49d9c976-0322-431a-88af-4c2dda8895fc"). InnerVolumeSpecName "kube-api-access-85cmp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 17:40:46.635524 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:46.635386    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49d9c976-0322-431a-88af-4c2dda8895fc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "49d9c976-0322-431a-88af-4c2dda8895fc" (UID: "49d9c976-0322-431a-88af-4c2dda8895fc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 17:40:46.635524 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:46.635389    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49d9c976-0322-431a-88af-4c2dda8895fc-dshm" (OuterVolumeSpecName: "dshm") pod "49d9c976-0322-431a-88af-4c2dda8895fc" (UID: "49d9c976-0322-431a-88af-4c2dda8895fc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:40:46.733916 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:46.733848    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49d9c976-0322-431a-88af-4c2dda8895fc-model-cache\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:40:46.733916 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:46.733910    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49d9c976-0322-431a-88af-4c2dda8895fc-tmp-dir\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:40:46.734134 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:46.733928    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-85cmp\" (UniqueName: \"kubernetes.io/projected/49d9c976-0322-431a-88af-4c2dda8895fc-kube-api-access-85cmp\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:40:46.734134 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:46.733942    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49d9c976-0322-431a-88af-4c2dda8895fc-home\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:40:46.734134 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:46.733954    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49d9c976-0322-431a-88af-4c2dda8895fc-tls-certs\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:40:46.734134 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:46.733966    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49d9c976-0322-431a-88af-4c2dda8895fc-dshm\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:40:46.788306 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:46.788223    2579 generic.go:356] "Generic (PLEG): container finished" podID="49d9c976-0322-431a-88af-4c2dda8895fc" containerID="80c95f3e000fb98357e45f9465412d727cd3d763091eadbaa0b8829c8d78663a" exitCode=0
Jul 07 17:40:46.788306 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:46.788266    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-567860/test-model-567860-kserve-59777bcd4f-xxl7c"
Jul 07 17:40:46.788538 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:46.788306    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-567860/test-model-567860-kserve-59777bcd4f-xxl7c" event={"ID":"49d9c976-0322-431a-88af-4c2dda8895fc","Type":"ContainerDied","Data":"80c95f3e000fb98357e45f9465412d727cd3d763091eadbaa0b8829c8d78663a"}
Jul 07 17:40:46.788538 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:46.788358    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-567860/test-model-567860-kserve-59777bcd4f-xxl7c" event={"ID":"49d9c976-0322-431a-88af-4c2dda8895fc","Type":"ContainerDied","Data":"16487391657f098c0c53e9ba7f35ecce3af11b383b9c8ad486d4039de763b79f"}
Jul 07 17:40:46.788538 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:46.788380    2579 scope.go:117] "RemoveContainer" containerID="80c95f3e000fb98357e45f9465412d727cd3d763091eadbaa0b8829c8d78663a"
Jul 07 17:40:46.797537 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:46.797518    2579 scope.go:117] "RemoveContainer" containerID="80c95f3e000fb98357e45f9465412d727cd3d763091eadbaa0b8829c8d78663a"
Jul 07 17:40:46.797804 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:40:46.797780    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80c95f3e000fb98357e45f9465412d727cd3d763091eadbaa0b8829c8d78663a\": container with ID starting with 80c95f3e000fb98357e45f9465412d727cd3d763091eadbaa0b8829c8d78663a not found: ID does not exist" containerID="80c95f3e000fb98357e45f9465412d727cd3d763091eadbaa0b8829c8d78663a"
Jul 07 17:40:46.797849 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:46.797817    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80c95f3e000fb98357e45f9465412d727cd3d763091eadbaa0b8829c8d78663a"} err="failed to get container status \"80c95f3e000fb98357e45f9465412d727cd3d763091eadbaa0b8829c8d78663a\": rpc error: code = NotFound desc = could not find container \"80c95f3e000fb98357e45f9465412d727cd3d763091eadbaa0b8829c8d78663a\": container with ID starting with 80c95f3e000fb98357e45f9465412d727cd3d763091eadbaa0b8829c8d78663a not found: ID does not exist"
Jul 07 17:40:46.811117 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:46.811084    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-567860/test-model-567860-kserve-59777bcd4f-xxl7c"]
Jul 07 17:40:46.815427 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:46.815407    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-567860/test-model-567860-kserve-59777bcd4f-xxl7c"]
Jul 07 17:40:47.027819 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:47.027779    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49d9c976-0322-431a-88af-4c2dda8895fc" path="/var/lib/kubelet/pods/49d9c976-0322-431a-88af-4c2dda8895fc/volumes"
Jul 07 17:40:47.028202 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:40:47.028188    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf32513a-2abf-402f-974b-6c75280d21a2" path="/var/lib/kubelet/pods/bf32513a-2abf-402f-974b-6c75280d21a2/volumes"
Jul 07 17:42:33.005124 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:33.005088    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-19c9b1/auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6"]
Jul 07 17:42:33.005758 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:33.005421    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf32513a-2abf-402f-974b-6c75280d21a2" containerName="main"
Jul 07 17:42:33.005758 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:33.005432    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf32513a-2abf-402f-974b-6c75280d21a2" containerName="main"
Jul 07 17:42:33.005758 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:33.005444    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49d9c976-0322-431a-88af-4c2dda8895fc" containerName="main"
Jul 07 17:42:33.005758 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:33.005450    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="49d9c976-0322-431a-88af-4c2dda8895fc" containerName="main"
Jul 07 17:42:33.005758 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:33.005508    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="49d9c976-0322-431a-88af-4c2dda8895fc" containerName="main"
Jul 07 17:42:33.005758 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:33.005515    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf32513a-2abf-402f-974b-6c75280d21a2" containerName="main"
Jul 07 17:42:33.008390 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:33.008374    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-19c9b1/auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6"
Jul 07 17:42:33.010658 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:33.010636    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-19c9b1\"/\"default-dockercfg-748gm\""
Jul 07 17:42:33.011574 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:33.011553    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-19c9b1\"/\"kube-root-ca.crt\""
Jul 07 17:42:33.011680 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:33.011584    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-19c9b1\"/\"auth-test-model-19c9b1-kserve-self-signed-certs\""
Jul 07 17:42:33.011680 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:33.011628    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-19c9b1\"/\"openshift-service-ca.crt\""
Jul 07 17:42:33.020790 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:33.020768    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-19c9b1/auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6"]
Jul 07 17:42:33.189204 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:33.189167    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4eec582-3d34-49f9-b8a1-6c08bf5efcc5-home\") pod \"auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6\" (UID: \"a4eec582-3d34-49f9-b8a1-6c08bf5efcc5\") " pod="ai-tenant-e2e-shared-a-19c9b1/auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6"
Jul 07 17:42:33.189204 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:33.189200    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4eec582-3d34-49f9-b8a1-6c08bf5efcc5-model-cache\") pod \"auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6\" (UID: \"a4eec582-3d34-49f9-b8a1-6c08bf5efcc5\") " pod="ai-tenant-e2e-shared-a-19c9b1/auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6"
Jul 07 17:42:33.189472 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:33.189223    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4eec582-3d34-49f9-b8a1-6c08bf5efcc5-tmp-dir\") pod \"auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6\" (UID: \"a4eec582-3d34-49f9-b8a1-6c08bf5efcc5\") " pod="ai-tenant-e2e-shared-a-19c9b1/auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6"
Jul 07 17:42:33.189472 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:33.189269    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4eec582-3d34-49f9-b8a1-6c08bf5efcc5-tls-certs\") pod \"auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6\" (UID: \"a4eec582-3d34-49f9-b8a1-6c08bf5efcc5\") " pod="ai-tenant-e2e-shared-a-19c9b1/auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6"
Jul 07 17:42:33.189472 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:33.189350    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4eec582-3d34-49f9-b8a1-6c08bf5efcc5-dshm\") pod \"auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6\" (UID: \"a4eec582-3d34-49f9-b8a1-6c08bf5efcc5\") " pod="ai-tenant-e2e-shared-a-19c9b1/auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6"
Jul 07 17:42:33.189472 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:33.189375    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wh8g4\" (UniqueName: \"kubernetes.io/projected/a4eec582-3d34-49f9-b8a1-6c08bf5efcc5-kube-api-access-wh8g4\") pod \"auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6\" (UID: \"a4eec582-3d34-49f9-b8a1-6c08bf5efcc5\") " pod="ai-tenant-e2e-shared-a-19c9b1/auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6"
Jul 07 17:42:33.289882 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:33.289810    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4eec582-3d34-49f9-b8a1-6c08bf5efcc5-home\") pod \"auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6\" (UID: \"a4eec582-3d34-49f9-b8a1-6c08bf5efcc5\") " pod="ai-tenant-e2e-shared-a-19c9b1/auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6"
Jul 07 17:42:33.289882 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:33.289839    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4eec582-3d34-49f9-b8a1-6c08bf5efcc5-model-cache\") pod \"auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6\" (UID: \"a4eec582-3d34-49f9-b8a1-6c08bf5efcc5\") " pod="ai-tenant-e2e-shared-a-19c9b1/auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6"
Jul 07 17:42:33.289882 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:33.289864    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4eec582-3d34-49f9-b8a1-6c08bf5efcc5-tmp-dir\") pod \"auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6\" (UID: \"a4eec582-3d34-49f9-b8a1-6c08bf5efcc5\") " pod="ai-tenant-e2e-shared-a-19c9b1/auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6"
Jul 07 17:42:33.289882 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:33.289878    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4eec582-3d34-49f9-b8a1-6c08bf5efcc5-tls-certs\") pod \"auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6\" (UID: \"a4eec582-3d34-49f9-b8a1-6c08bf5efcc5\") " pod="ai-tenant-e2e-shared-a-19c9b1/auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6"
Jul 07 17:42:33.290302 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:33.289925    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4eec582-3d34-49f9-b8a1-6c08bf5efcc5-dshm\") pod \"auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6\" (UID: \"a4eec582-3d34-49f9-b8a1-6c08bf5efcc5\") " pod="ai-tenant-e2e-shared-a-19c9b1/auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6"
Jul 07 17:42:33.290302 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:33.289949    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wh8g4\" (UniqueName: \"kubernetes.io/projected/a4eec582-3d34-49f9-b8a1-6c08bf5efcc5-kube-api-access-wh8g4\") pod \"auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6\" (UID: \"a4eec582-3d34-49f9-b8a1-6c08bf5efcc5\") " pod="ai-tenant-e2e-shared-a-19c9b1/auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6"
Jul 07 17:42:33.290302 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:33.290198    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4eec582-3d34-49f9-b8a1-6c08bf5efcc5-home\") pod \"auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6\" (UID: \"a4eec582-3d34-49f9-b8a1-6c08bf5efcc5\") " pod="ai-tenant-e2e-shared-a-19c9b1/auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6"
Jul 07 17:42:33.290302 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:33.290260    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4eec582-3d34-49f9-b8a1-6c08bf5efcc5-tmp-dir\") pod \"auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6\" (UID: \"a4eec582-3d34-49f9-b8a1-6c08bf5efcc5\") " pod="ai-tenant-e2e-shared-a-19c9b1/auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6"
Jul 07 17:42:33.290466 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:33.290308    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4eec582-3d34-49f9-b8a1-6c08bf5efcc5-model-cache\") pod \"auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6\" (UID: \"a4eec582-3d34-49f9-b8a1-6c08bf5efcc5\") " pod="ai-tenant-e2e-shared-a-19c9b1/auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6"
Jul 07 17:42:33.292288 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:33.292266    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4eec582-3d34-49f9-b8a1-6c08bf5efcc5-dshm\") pod \"auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6\" (UID: \"a4eec582-3d34-49f9-b8a1-6c08bf5efcc5\") " pod="ai-tenant-e2e-shared-a-19c9b1/auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6"
Jul 07 17:42:33.292448 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:33.292431    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4eec582-3d34-49f9-b8a1-6c08bf5efcc5-tls-certs\") pod \"auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6\" (UID: \"a4eec582-3d34-49f9-b8a1-6c08bf5efcc5\") " pod="ai-tenant-e2e-shared-a-19c9b1/auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6"
Jul 07 17:42:33.299943 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:33.299922    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wh8g4\" (UniqueName: \"kubernetes.io/projected/a4eec582-3d34-49f9-b8a1-6c08bf5efcc5-kube-api-access-wh8g4\") pod \"auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6\" (UID: \"a4eec582-3d34-49f9-b8a1-6c08bf5efcc5\") " pod="ai-tenant-e2e-shared-a-19c9b1/auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6"
Jul 07 17:42:33.317704 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:33.317685    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-19c9b1/auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6"
Jul 07 17:42:33.450601 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:33.450576    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-19c9b1/auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6"]
Jul 07 17:42:33.452278 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:42:33.452245    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda4eec582_3d34_49f9_b8a1_6c08bf5efcc5.slice/crio-b7b1f204a0574458074522af3abb1f805b6bc1c993aa44eb71e6504e5249f887 WatchSource:0}: Error finding container b7b1f204a0574458074522af3abb1f805b6bc1c993aa44eb71e6504e5249f887: Status 404 returned error can't find the container with id b7b1f204a0574458074522af3abb1f805b6bc1c993aa44eb71e6504e5249f887
Jul 07 17:42:34.335421 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:34.335381    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-19c9b1/auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6" event={"ID":"a4eec582-3d34-49f9-b8a1-6c08bf5efcc5","Type":"ContainerStarted","Data":"8a1ecdbef0f3723bfcccea777ea45c698bc21f56ea538f07a67ab2dcf762bc45"}
Jul 07 17:42:34.335421 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:34.335428    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-19c9b1/auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6" event={"ID":"a4eec582-3d34-49f9-b8a1-6c08bf5efcc5","Type":"ContainerStarted","Data":"b7b1f204a0574458074522af3abb1f805b6bc1c993aa44eb71e6504e5249f887"}
Jul 07 17:42:34.352985 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:34.352918    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-19c9b1/auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6" podStartSLOduration=2.3528738860000002 podStartE2EDuration="2.352873886s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:42:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:42:34.352716386 +0000 UTC m=+2077.848794895" watchObservedRunningTime="2026-07-07 17:42:34.352873886 +0000 UTC m=+2077.848952395"
Jul 07 17:42:43.318396 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:43.318362    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-19c9b1/auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6"
Jul 07 17:42:43.318396 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:43.318402    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-19c9b1/auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6"
Jul 07 17:42:43.331005 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:43.330979    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-19c9b1/auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6"
Jul 07 17:42:43.399097 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:43.399064    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-19c9b1/auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6"
Jul 07 17:42:59.472561 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:59.472525    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-ce87c9/auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv"]
Jul 07 17:42:59.476221 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:59.476198    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ce87c9/auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv"
Jul 07 17:42:59.479483 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:59.479459    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ce87c9\"/\"auth-test-model-ce87c9-kserve-self-signed-certs\""
Jul 07 17:42:59.479483 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:59.479471    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ce87c9\"/\"openshift-service-ca.crt\""
Jul 07 17:42:59.479703 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:59.479501    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ce87c9\"/\"default-dockercfg-9qmxp\""
Jul 07 17:42:59.479703 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:59.479641    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ce87c9\"/\"kube-root-ca.crt\""
Jul 07 17:42:59.488622 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:59.488598    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ce87c9/auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv"]
Jul 07 17:42:59.523439 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:59.523400    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7768cc9c-a345-44e4-a564-3688a3c595e0-home\") pod \"auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv\" (UID: \"7768cc9c-a345-44e4-a564-3688a3c595e0\") " pod="ai-tenant-e2e-shared-b-ce87c9/auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv"
Jul 07 17:42:59.523636 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:59.523461    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7768cc9c-a345-44e4-a564-3688a3c595e0-dshm\") pod \"auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv\" (UID: \"7768cc9c-a345-44e4-a564-3688a3c595e0\") " pod="ai-tenant-e2e-shared-b-ce87c9/auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv"
Jul 07 17:42:59.523636 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:59.523527    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7768cc9c-a345-44e4-a564-3688a3c595e0-tls-certs\") pod \"auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv\" (UID: \"7768cc9c-a345-44e4-a564-3688a3c595e0\") " pod="ai-tenant-e2e-shared-b-ce87c9/auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv"
Jul 07 17:42:59.523636 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:59.523590    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7768cc9c-a345-44e4-a564-3688a3c595e0-model-cache\") pod \"auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv\" (UID: \"7768cc9c-a345-44e4-a564-3688a3c595e0\") " pod="ai-tenant-e2e-shared-b-ce87c9/auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv"
Jul 07 17:42:59.523636 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:59.523632    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rdjk\" (UniqueName: \"kubernetes.io/projected/7768cc9c-a345-44e4-a564-3688a3c595e0-kube-api-access-7rdjk\") pod \"auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv\" (UID: \"7768cc9c-a345-44e4-a564-3688a3c595e0\") " pod="ai-tenant-e2e-shared-b-ce87c9/auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv"
Jul 07 17:42:59.523777 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:59.523677    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7768cc9c-a345-44e4-a564-3688a3c595e0-tmp-dir\") pod \"auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv\" (UID: \"7768cc9c-a345-44e4-a564-3688a3c595e0\") " pod="ai-tenant-e2e-shared-b-ce87c9/auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv"
Jul 07 17:42:59.624228 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:59.624185    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7768cc9c-a345-44e4-a564-3688a3c595e0-dshm\") pod \"auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv\" (UID: \"7768cc9c-a345-44e4-a564-3688a3c595e0\") " pod="ai-tenant-e2e-shared-b-ce87c9/auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv"
Jul 07 17:42:59.624444 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:59.624237    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7768cc9c-a345-44e4-a564-3688a3c595e0-tls-certs\") pod \"auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv\" (UID: \"7768cc9c-a345-44e4-a564-3688a3c595e0\") " pod="ai-tenant-e2e-shared-b-ce87c9/auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv"
Jul 07 17:42:59.624444 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:59.624275    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7768cc9c-a345-44e4-a564-3688a3c595e0-model-cache\") pod \"auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv\" (UID: \"7768cc9c-a345-44e4-a564-3688a3c595e0\") " pod="ai-tenant-e2e-shared-b-ce87c9/auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv"
Jul 07 17:42:59.624444 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:59.624310    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rdjk\" (UniqueName: \"kubernetes.io/projected/7768cc9c-a345-44e4-a564-3688a3c595e0-kube-api-access-7rdjk\") pod \"auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv\" (UID: \"7768cc9c-a345-44e4-a564-3688a3c595e0\") " pod="ai-tenant-e2e-shared-b-ce87c9/auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv"
Jul 07 17:42:59.624444 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:59.624360    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7768cc9c-a345-44e4-a564-3688a3c595e0-tmp-dir\") pod \"auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv\" (UID: \"7768cc9c-a345-44e4-a564-3688a3c595e0\") " pod="ai-tenant-e2e-shared-b-ce87c9/auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv"
Jul 07 17:42:59.624444 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:59.624425    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7768cc9c-a345-44e4-a564-3688a3c595e0-home\") pod \"auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv\" (UID: \"7768cc9c-a345-44e4-a564-3688a3c595e0\") " pod="ai-tenant-e2e-shared-b-ce87c9/auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv"
Jul 07 17:42:59.624717 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:59.624681    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7768cc9c-a345-44e4-a564-3688a3c595e0-model-cache\") pod \"auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv\" (UID: \"7768cc9c-a345-44e4-a564-3688a3c595e0\") " pod="ai-tenant-e2e-shared-b-ce87c9/auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv"
Jul 07 17:42:59.624779 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:59.624732    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7768cc9c-a345-44e4-a564-3688a3c595e0-tmp-dir\") pod \"auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv\" (UID: \"7768cc9c-a345-44e4-a564-3688a3c595e0\") " pod="ai-tenant-e2e-shared-b-ce87c9/auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv"
Jul 07 17:42:59.624837 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:59.624798    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7768cc9c-a345-44e4-a564-3688a3c595e0-home\") pod \"auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv\" (UID: \"7768cc9c-a345-44e4-a564-3688a3c595e0\") " pod="ai-tenant-e2e-shared-b-ce87c9/auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv"
Jul 07 17:42:59.626689 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:59.626663    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7768cc9c-a345-44e4-a564-3688a3c595e0-dshm\") pod \"auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv\" (UID: \"7768cc9c-a345-44e4-a564-3688a3c595e0\") " pod="ai-tenant-e2e-shared-b-ce87c9/auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv"
Jul 07 17:42:59.627026 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:59.627009    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7768cc9c-a345-44e4-a564-3688a3c595e0-tls-certs\") pod \"auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv\" (UID: \"7768cc9c-a345-44e4-a564-3688a3c595e0\") " pod="ai-tenant-e2e-shared-b-ce87c9/auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv"
Jul 07 17:42:59.636572 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:59.636539    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rdjk\" (UniqueName: \"kubernetes.io/projected/7768cc9c-a345-44e4-a564-3688a3c595e0-kube-api-access-7rdjk\") pod \"auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv\" (UID: \"7768cc9c-a345-44e4-a564-3688a3c595e0\") " pod="ai-tenant-e2e-shared-b-ce87c9/auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv"
Jul 07 17:42:59.785742 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:59.785641    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ce87c9/auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv"
Jul 07 17:42:59.937090 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:42:59.936930    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ce87c9/auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv"]
Jul 07 17:42:59.939327 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:42:59.939294    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7768cc9c_a345_44e4_a564_3688a3c595e0.slice/crio-b19d4208cfe67c907df5de8fd45ce69652ba541cbadd0d818414ddea97a1bd8d WatchSource:0}: Error finding container b19d4208cfe67c907df5de8fd45ce69652ba541cbadd0d818414ddea97a1bd8d: Status 404 returned error can't find the container with id b19d4208cfe67c907df5de8fd45ce69652ba541cbadd0d818414ddea97a1bd8d
Jul 07 17:43:00.478426 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:43:00.478392    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ce87c9/auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv" event={"ID":"7768cc9c-a345-44e4-a564-3688a3c595e0","Type":"ContainerStarted","Data":"1fb05813f0cf3b08dce3e2730020a437e048327aa6d781a9a10817e1889d6c68"}
Jul 07 17:43:00.478843 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:43:00.478443    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ce87c9/auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv" event={"ID":"7768cc9c-a345-44e4-a564-3688a3c595e0","Type":"ContainerStarted","Data":"b19d4208cfe67c907df5de8fd45ce69652ba541cbadd0d818414ddea97a1bd8d"}
Jul 07 17:43:00.499823 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:43:00.499761    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-ce87c9/auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv" podStartSLOduration=1.499743268 podStartE2EDuration="1.499743268s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:42:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:43:00.49875774 +0000 UTC m=+2103.994836248" watchObservedRunningTime="2026-07-07 17:43:00.499743268 +0000 UTC m=+2103.995821775"
Jul 07 17:43:09.786406 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:43:09.786358    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-ce87c9/auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv"
Jul 07 17:43:09.786865 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:43:09.786426    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-ce87c9/auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv"
Jul 07 17:43:09.799495 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:43:09.799403    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-ce87c9/auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv"
Jul 07 17:43:10.544203 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:43:10.544168    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-ce87c9/auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv"
Jul 07 17:45:00.410594 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:00.410562    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-19c9b1/auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6"]
Jul 07 17:45:01.358658 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:01.358616    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ce87c9/auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv"]
Jul 07 17:45:10.289416 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:10.289375    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-19c9b1/sub-test-model-19c9b1-kserve-56787b8b45-rk96c"]
Jul 07 17:45:10.293293 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:10.293272    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-19c9b1/sub-test-model-19c9b1-kserve-56787b8b45-rk96c"
Jul 07 17:45:10.295592 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:10.295567    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-19c9b1\"/\"sub-test-model-19c9b1-kserve-self-signed-certs\""
Jul 07 17:45:10.301670 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:10.301645    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-19c9b1/sub-test-model-19c9b1-kserve-56787b8b45-rk96c"]
Jul 07 17:45:10.388918 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:10.388856    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1449b9a5-4318-4df3-813d-4cfd35ae88a3-home\") pod \"sub-test-model-19c9b1-kserve-56787b8b45-rk96c\" (UID: \"1449b9a5-4318-4df3-813d-4cfd35ae88a3\") " pod="ai-tenant-e2e-shared-a-19c9b1/sub-test-model-19c9b1-kserve-56787b8b45-rk96c"
Jul 07 17:45:10.388918 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:10.388922    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1449b9a5-4318-4df3-813d-4cfd35ae88a3-dshm\") pod \"sub-test-model-19c9b1-kserve-56787b8b45-rk96c\" (UID: \"1449b9a5-4318-4df3-813d-4cfd35ae88a3\") " pod="ai-tenant-e2e-shared-a-19c9b1/sub-test-model-19c9b1-kserve-56787b8b45-rk96c"
Jul 07 17:45:10.389128 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:10.388995    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1449b9a5-4318-4df3-813d-4cfd35ae88a3-tls-certs\") pod \"sub-test-model-19c9b1-kserve-56787b8b45-rk96c\" (UID: \"1449b9a5-4318-4df3-813d-4cfd35ae88a3\") " pod="ai-tenant-e2e-shared-a-19c9b1/sub-test-model-19c9b1-kserve-56787b8b45-rk96c"
Jul 07 17:45:10.389128 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:10.389031    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1449b9a5-4318-4df3-813d-4cfd35ae88a3-model-cache\") pod \"sub-test-model-19c9b1-kserve-56787b8b45-rk96c\" (UID: \"1449b9a5-4318-4df3-813d-4cfd35ae88a3\") " pod="ai-tenant-e2e-shared-a-19c9b1/sub-test-model-19c9b1-kserve-56787b8b45-rk96c"
Jul 07 17:45:10.389128 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:10.389047    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4btmp\" (UniqueName: \"kubernetes.io/projected/1449b9a5-4318-4df3-813d-4cfd35ae88a3-kube-api-access-4btmp\") pod \"sub-test-model-19c9b1-kserve-56787b8b45-rk96c\" (UID: \"1449b9a5-4318-4df3-813d-4cfd35ae88a3\") " pod="ai-tenant-e2e-shared-a-19c9b1/sub-test-model-19c9b1-kserve-56787b8b45-rk96c"
Jul 07 17:45:10.389128 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:10.389104    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1449b9a5-4318-4df3-813d-4cfd35ae88a3-tmp-dir\") pod \"sub-test-model-19c9b1-kserve-56787b8b45-rk96c\" (UID: \"1449b9a5-4318-4df3-813d-4cfd35ae88a3\") " pod="ai-tenant-e2e-shared-a-19c9b1/sub-test-model-19c9b1-kserve-56787b8b45-rk96c"
Jul 07 17:45:10.489843 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:10.489817    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1449b9a5-4318-4df3-813d-4cfd35ae88a3-tmp-dir\") pod \"sub-test-model-19c9b1-kserve-56787b8b45-rk96c\" (UID: \"1449b9a5-4318-4df3-813d-4cfd35ae88a3\") " pod="ai-tenant-e2e-shared-a-19c9b1/sub-test-model-19c9b1-kserve-56787b8b45-rk96c"
Jul 07 17:45:10.490049 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:10.489870    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1449b9a5-4318-4df3-813d-4cfd35ae88a3-home\") pod \"sub-test-model-19c9b1-kserve-56787b8b45-rk96c\" (UID: \"1449b9a5-4318-4df3-813d-4cfd35ae88a3\") " pod="ai-tenant-e2e-shared-a-19c9b1/sub-test-model-19c9b1-kserve-56787b8b45-rk96c"
Jul 07 17:45:10.490049 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:10.489918    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1449b9a5-4318-4df3-813d-4cfd35ae88a3-dshm\") pod \"sub-test-model-19c9b1-kserve-56787b8b45-rk96c\" (UID: \"1449b9a5-4318-4df3-813d-4cfd35ae88a3\") " pod="ai-tenant-e2e-shared-a-19c9b1/sub-test-model-19c9b1-kserve-56787b8b45-rk96c"
Jul 07 17:45:10.490049 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:10.489949    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1449b9a5-4318-4df3-813d-4cfd35ae88a3-tls-certs\") pod \"sub-test-model-19c9b1-kserve-56787b8b45-rk96c\" (UID: \"1449b9a5-4318-4df3-813d-4cfd35ae88a3\") " pod="ai-tenant-e2e-shared-a-19c9b1/sub-test-model-19c9b1-kserve-56787b8b45-rk96c"
Jul 07 17:45:10.490049 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:10.489967    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1449b9a5-4318-4df3-813d-4cfd35ae88a3-model-cache\") pod \"sub-test-model-19c9b1-kserve-56787b8b45-rk96c\" (UID: \"1449b9a5-4318-4df3-813d-4cfd35ae88a3\") " pod="ai-tenant-e2e-shared-a-19c9b1/sub-test-model-19c9b1-kserve-56787b8b45-rk96c"
Jul 07 17:45:10.490049 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:10.489988    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4btmp\" (UniqueName: \"kubernetes.io/projected/1449b9a5-4318-4df3-813d-4cfd35ae88a3-kube-api-access-4btmp\") pod \"sub-test-model-19c9b1-kserve-56787b8b45-rk96c\" (UID: \"1449b9a5-4318-4df3-813d-4cfd35ae88a3\") " pod="ai-tenant-e2e-shared-a-19c9b1/sub-test-model-19c9b1-kserve-56787b8b45-rk96c"
Jul 07 17:45:10.490343 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:10.490229    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1449b9a5-4318-4df3-813d-4cfd35ae88a3-tmp-dir\") pod \"sub-test-model-19c9b1-kserve-56787b8b45-rk96c\" (UID: \"1449b9a5-4318-4df3-813d-4cfd35ae88a3\") " pod="ai-tenant-e2e-shared-a-19c9b1/sub-test-model-19c9b1-kserve-56787b8b45-rk96c"
Jul 07 17:45:10.490343 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:10.490334    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1449b9a5-4318-4df3-813d-4cfd35ae88a3-model-cache\") pod \"sub-test-model-19c9b1-kserve-56787b8b45-rk96c\" (UID: \"1449b9a5-4318-4df3-813d-4cfd35ae88a3\") " pod="ai-tenant-e2e-shared-a-19c9b1/sub-test-model-19c9b1-kserve-56787b8b45-rk96c"
Jul 07 17:45:10.490435 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:10.490365    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1449b9a5-4318-4df3-813d-4cfd35ae88a3-home\") pod \"sub-test-model-19c9b1-kserve-56787b8b45-rk96c\" (UID: \"1449b9a5-4318-4df3-813d-4cfd35ae88a3\") " pod="ai-tenant-e2e-shared-a-19c9b1/sub-test-model-19c9b1-kserve-56787b8b45-rk96c"
Jul 07 17:45:10.492307 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:10.492286    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1449b9a5-4318-4df3-813d-4cfd35ae88a3-dshm\") pod \"sub-test-model-19c9b1-kserve-56787b8b45-rk96c\" (UID: \"1449b9a5-4318-4df3-813d-4cfd35ae88a3\") " pod="ai-tenant-e2e-shared-a-19c9b1/sub-test-model-19c9b1-kserve-56787b8b45-rk96c"
Jul 07 17:45:10.492573 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:10.492556    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1449b9a5-4318-4df3-813d-4cfd35ae88a3-tls-certs\") pod \"sub-test-model-19c9b1-kserve-56787b8b45-rk96c\" (UID: \"1449b9a5-4318-4df3-813d-4cfd35ae88a3\") " pod="ai-tenant-e2e-shared-a-19c9b1/sub-test-model-19c9b1-kserve-56787b8b45-rk96c"
Jul 07 17:45:10.499425 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:10.499400    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4btmp\" (UniqueName: \"kubernetes.io/projected/1449b9a5-4318-4df3-813d-4cfd35ae88a3-kube-api-access-4btmp\") pod \"sub-test-model-19c9b1-kserve-56787b8b45-rk96c\" (UID: \"1449b9a5-4318-4df3-813d-4cfd35ae88a3\") " pod="ai-tenant-e2e-shared-a-19c9b1/sub-test-model-19c9b1-kserve-56787b8b45-rk96c"
Jul 07 17:45:10.603021 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:10.602949    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-19c9b1/sub-test-model-19c9b1-kserve-56787b8b45-rk96c"
Jul 07 17:45:10.742735 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:10.742713    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-19c9b1/sub-test-model-19c9b1-kserve-56787b8b45-rk96c"]
Jul 07 17:45:10.743634 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:45:10.743611    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1449b9a5_4318_4df3_813d_4cfd35ae88a3.slice/crio-1170056ac0080bf59926f1dc9129ce12c8fb7e0c80df92dba37d2a7553c16b89 WatchSource:0}: Error finding container 1170056ac0080bf59926f1dc9129ce12c8fb7e0c80df92dba37d2a7553c16b89: Status 404 returned error can't find the container with id 1170056ac0080bf59926f1dc9129ce12c8fb7e0c80df92dba37d2a7553c16b89
Jul 07 17:45:11.164595 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:11.164561    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-19c9b1/sub-test-model-19c9b1-kserve-56787b8b45-rk96c" event={"ID":"1449b9a5-4318-4df3-813d-4cfd35ae88a3","Type":"ContainerStarted","Data":"e9a41cbe2bb2ca4276573b283676030e4f7a4b3c0b9eb836f351faad9f3d0ec6"}
Jul 07 17:45:11.164595 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:11.164600    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-19c9b1/sub-test-model-19c9b1-kserve-56787b8b45-rk96c" event={"ID":"1449b9a5-4318-4df3-813d-4cfd35ae88a3","Type":"ContainerStarted","Data":"1170056ac0080bf59926f1dc9129ce12c8fb7e0c80df92dba37d2a7553c16b89"}
Jul 07 17:45:11.184413 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:11.184370    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-19c9b1/sub-test-model-19c9b1-kserve-56787b8b45-rk96c" podStartSLOduration=1.184356778 podStartE2EDuration="1.184356778s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:45:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:45:11.183347029 +0000 UTC m=+2234.679425537" watchObservedRunningTime="2026-07-07 17:45:11.184356778 +0000 UTC m=+2234.680435301"
Jul 07 17:45:15.431620 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:15.431581    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-19c9b1/auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6" podUID="a4eec582-3d34-49f9-b8a1-6c08bf5efcc5" containerName="main" containerID="cri-o://8a1ecdbef0f3723bfcccea777ea45c698bc21f56ea538f07a67ab2dcf762bc45" gracePeriod=45
Jul 07 17:45:15.709357 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:15.709333    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-19c9b1/auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6"
Jul 07 17:45:15.836558 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:15.836525    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4eec582-3d34-49f9-b8a1-6c08bf5efcc5-tmp-dir\") pod \"a4eec582-3d34-49f9-b8a1-6c08bf5efcc5\" (UID: \"a4eec582-3d34-49f9-b8a1-6c08bf5efcc5\") "
Jul 07 17:45:15.836746 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:15.836573    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4eec582-3d34-49f9-b8a1-6c08bf5efcc5-model-cache\") pod \"a4eec582-3d34-49f9-b8a1-6c08bf5efcc5\" (UID: \"a4eec582-3d34-49f9-b8a1-6c08bf5efcc5\") "
Jul 07 17:45:15.836746 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:15.836642    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4eec582-3d34-49f9-b8a1-6c08bf5efcc5-home\") pod \"a4eec582-3d34-49f9-b8a1-6c08bf5efcc5\" (UID: \"a4eec582-3d34-49f9-b8a1-6c08bf5efcc5\") "
Jul 07 17:45:15.836746 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:15.836668    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4eec582-3d34-49f9-b8a1-6c08bf5efcc5-tls-certs\") pod \"a4eec582-3d34-49f9-b8a1-6c08bf5efcc5\" (UID: \"a4eec582-3d34-49f9-b8a1-6c08bf5efcc5\") "
Jul 07 17:45:15.836746 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:15.836692    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wh8g4\" (UniqueName: \"kubernetes.io/projected/a4eec582-3d34-49f9-b8a1-6c08bf5efcc5-kube-api-access-wh8g4\") pod \"a4eec582-3d34-49f9-b8a1-6c08bf5efcc5\" (UID: \"a4eec582-3d34-49f9-b8a1-6c08bf5efcc5\") "
Jul 07 17:45:15.836746 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:15.836723    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4eec582-3d34-49f9-b8a1-6c08bf5efcc5-dshm\") pod \"a4eec582-3d34-49f9-b8a1-6c08bf5efcc5\" (UID: \"a4eec582-3d34-49f9-b8a1-6c08bf5efcc5\") "
Jul 07 17:45:15.837045 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:15.836756    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4eec582-3d34-49f9-b8a1-6c08bf5efcc5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a4eec582-3d34-49f9-b8a1-6c08bf5efcc5" (UID: "a4eec582-3d34-49f9-b8a1-6c08bf5efcc5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:45:15.837045 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:15.836846    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4eec582-3d34-49f9-b8a1-6c08bf5efcc5-model-cache" (OuterVolumeSpecName: "model-cache") pod "a4eec582-3d34-49f9-b8a1-6c08bf5efcc5" (UID: "a4eec582-3d34-49f9-b8a1-6c08bf5efcc5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:45:15.837045 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:15.836944    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4eec582-3d34-49f9-b8a1-6c08bf5efcc5-home" (OuterVolumeSpecName: "home") pod "a4eec582-3d34-49f9-b8a1-6c08bf5efcc5" (UID: "a4eec582-3d34-49f9-b8a1-6c08bf5efcc5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:45:15.837214 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:15.837088    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4eec582-3d34-49f9-b8a1-6c08bf5efcc5-tmp-dir\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:45:15.837214 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:15.837105    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4eec582-3d34-49f9-b8a1-6c08bf5efcc5-model-cache\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:45:15.837214 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:15.837118    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4eec582-3d34-49f9-b8a1-6c08bf5efcc5-home\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:45:15.839024 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:15.838998    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4eec582-3d34-49f9-b8a1-6c08bf5efcc5-dshm" (OuterVolumeSpecName: "dshm") pod "a4eec582-3d34-49f9-b8a1-6c08bf5efcc5" (UID: "a4eec582-3d34-49f9-b8a1-6c08bf5efcc5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:45:15.839133 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:15.839018    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4eec582-3d34-49f9-b8a1-6c08bf5efcc5-kube-api-access-wh8g4" (OuterVolumeSpecName: "kube-api-access-wh8g4") pod "a4eec582-3d34-49f9-b8a1-6c08bf5efcc5" (UID: "a4eec582-3d34-49f9-b8a1-6c08bf5efcc5"). InnerVolumeSpecName "kube-api-access-wh8g4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 17:45:15.839133 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:15.839036    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4eec582-3d34-49f9-b8a1-6c08bf5efcc5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a4eec582-3d34-49f9-b8a1-6c08bf5efcc5" (UID: "a4eec582-3d34-49f9-b8a1-6c08bf5efcc5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 17:45:15.937664 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:15.937630    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4eec582-3d34-49f9-b8a1-6c08bf5efcc5-tls-certs\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:45:15.937664 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:15.937659    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wh8g4\" (UniqueName: \"kubernetes.io/projected/a4eec582-3d34-49f9-b8a1-6c08bf5efcc5-kube-api-access-wh8g4\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:45:15.937664 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:15.937670    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4eec582-3d34-49f9-b8a1-6c08bf5efcc5-dshm\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:45:16.192657 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:16.192626    2579 generic.go:356] "Generic (PLEG): container finished" podID="a4eec582-3d34-49f9-b8a1-6c08bf5efcc5" containerID="8a1ecdbef0f3723bfcccea777ea45c698bc21f56ea538f07a67ab2dcf762bc45" exitCode=0
Jul 07 17:45:16.192829 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:16.192682    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-19c9b1/auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6"
Jul 07 17:45:16.192829 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:16.192722    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-19c9b1/auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6" event={"ID":"a4eec582-3d34-49f9-b8a1-6c08bf5efcc5","Type":"ContainerDied","Data":"8a1ecdbef0f3723bfcccea777ea45c698bc21f56ea538f07a67ab2dcf762bc45"}
Jul 07 17:45:16.192829 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:16.192766    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-19c9b1/auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6" event={"ID":"a4eec582-3d34-49f9-b8a1-6c08bf5efcc5","Type":"ContainerDied","Data":"b7b1f204a0574458074522af3abb1f805b6bc1c993aa44eb71e6504e5249f887"}
Jul 07 17:45:16.192829 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:16.192789    2579 scope.go:117] "RemoveContainer" containerID="8a1ecdbef0f3723bfcccea777ea45c698bc21f56ea538f07a67ab2dcf762bc45"
Jul 07 17:45:16.201640 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:16.201625    2579 scope.go:117] "RemoveContainer" containerID="8a1ecdbef0f3723bfcccea777ea45c698bc21f56ea538f07a67ab2dcf762bc45"
Jul 07 17:45:16.201907 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:45:16.201875    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a1ecdbef0f3723bfcccea777ea45c698bc21f56ea538f07a67ab2dcf762bc45\": container with ID starting with 8a1ecdbef0f3723bfcccea777ea45c698bc21f56ea538f07a67ab2dcf762bc45 not found: ID does not exist" containerID="8a1ecdbef0f3723bfcccea777ea45c698bc21f56ea538f07a67ab2dcf762bc45"
Jul 07 17:45:16.201991 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:16.201915    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a1ecdbef0f3723bfcccea777ea45c698bc21f56ea538f07a67ab2dcf762bc45"} err="failed to get container status \"8a1ecdbef0f3723bfcccea777ea45c698bc21f56ea538f07a67ab2dcf762bc45\": rpc error: code = NotFound desc = could not find container \"8a1ecdbef0f3723bfcccea777ea45c698bc21f56ea538f07a67ab2dcf762bc45\": container with ID starting with 8a1ecdbef0f3723bfcccea777ea45c698bc21f56ea538f07a67ab2dcf762bc45 not found: ID does not exist"
Jul 07 17:45:16.216206 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:16.216185    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-19c9b1/auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6"]
Jul 07 17:45:16.222774 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:16.222755    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-19c9b1/auth-test-model-19c9b1-kserve-57944c8c9f-cx9t6"]
Jul 07 17:45:16.378998 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:16.378951    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-ce87c9/auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv" podUID="7768cc9c-a345-44e4-a564-3688a3c595e0" containerName="main" containerID="cri-o://1fb05813f0cf3b08dce3e2730020a437e048327aa6d781a9a10817e1889d6c68" gracePeriod=45
Jul 07 17:45:16.633053 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:16.633028    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ce87c9/auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv"
Jul 07 17:45:16.745467 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:16.745386    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7768cc9c-a345-44e4-a564-3688a3c595e0-model-cache\") pod \"7768cc9c-a345-44e4-a564-3688a3c595e0\" (UID: \"7768cc9c-a345-44e4-a564-3688a3c595e0\") "
Jul 07 17:45:16.745603 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:16.745482    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7768cc9c-a345-44e4-a564-3688a3c595e0-tmp-dir\") pod \"7768cc9c-a345-44e4-a564-3688a3c595e0\" (UID: \"7768cc9c-a345-44e4-a564-3688a3c595e0\") "
Jul 07 17:45:16.745603 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:16.745517    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7rdjk\" (UniqueName: \"kubernetes.io/projected/7768cc9c-a345-44e4-a564-3688a3c595e0-kube-api-access-7rdjk\") pod \"7768cc9c-a345-44e4-a564-3688a3c595e0\" (UID: \"7768cc9c-a345-44e4-a564-3688a3c595e0\") "
Jul 07 17:45:16.745603 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:16.745544    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7768cc9c-a345-44e4-a564-3688a3c595e0-home\") pod \"7768cc9c-a345-44e4-a564-3688a3c595e0\" (UID: \"7768cc9c-a345-44e4-a564-3688a3c595e0\") "
Jul 07 17:45:16.745603 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:16.745579    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7768cc9c-a345-44e4-a564-3688a3c595e0-tls-certs\") pod \"7768cc9c-a345-44e4-a564-3688a3c595e0\" (UID: \"7768cc9c-a345-44e4-a564-3688a3c595e0\") "
Jul 07 17:45:16.745757 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:16.745654    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7768cc9c-a345-44e4-a564-3688a3c595e0-dshm\") pod \"7768cc9c-a345-44e4-a564-3688a3c595e0\" (UID: \"7768cc9c-a345-44e4-a564-3688a3c595e0\") "
Jul 07 17:45:16.745757 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:16.745673    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7768cc9c-a345-44e4-a564-3688a3c595e0-model-cache" (OuterVolumeSpecName: "model-cache") pod "7768cc9c-a345-44e4-a564-3688a3c595e0" (UID: "7768cc9c-a345-44e4-a564-3688a3c595e0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:45:16.745757 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:16.745709    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7768cc9c-a345-44e4-a564-3688a3c595e0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7768cc9c-a345-44e4-a564-3688a3c595e0" (UID: "7768cc9c-a345-44e4-a564-3688a3c595e0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:45:16.745937 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:16.745776    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7768cc9c-a345-44e4-a564-3688a3c595e0-home" (OuterVolumeSpecName: "home") pod "7768cc9c-a345-44e4-a564-3688a3c595e0" (UID: "7768cc9c-a345-44e4-a564-3688a3c595e0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:45:16.746003 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:16.745954    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7768cc9c-a345-44e4-a564-3688a3c595e0-model-cache\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:45:16.746003 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:16.745968    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7768cc9c-a345-44e4-a564-3688a3c595e0-tmp-dir\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:45:16.746003 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:16.745976    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7768cc9c-a345-44e4-a564-3688a3c595e0-home\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:45:16.747785 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:16.747756    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7768cc9c-a345-44e4-a564-3688a3c595e0-dshm" (OuterVolumeSpecName: "dshm") pod "7768cc9c-a345-44e4-a564-3688a3c595e0" (UID: "7768cc9c-a345-44e4-a564-3688a3c595e0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:45:16.747882 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:16.747786    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7768cc9c-a345-44e4-a564-3688a3c595e0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7768cc9c-a345-44e4-a564-3688a3c595e0" (UID: "7768cc9c-a345-44e4-a564-3688a3c595e0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 17:45:16.747882 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:16.747835    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7768cc9c-a345-44e4-a564-3688a3c595e0-kube-api-access-7rdjk" (OuterVolumeSpecName: "kube-api-access-7rdjk") pod "7768cc9c-a345-44e4-a564-3688a3c595e0" (UID: "7768cc9c-a345-44e4-a564-3688a3c595e0"). InnerVolumeSpecName "kube-api-access-7rdjk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 17:45:16.847155 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:16.847122    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7rdjk\" (UniqueName: \"kubernetes.io/projected/7768cc9c-a345-44e4-a564-3688a3c595e0-kube-api-access-7rdjk\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:45:16.847155 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:16.847153    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7768cc9c-a345-44e4-a564-3688a3c595e0-tls-certs\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:45:16.847155 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:16.847161    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7768cc9c-a345-44e4-a564-3688a3c595e0-dshm\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:45:17.027488 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:17.027417    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4eec582-3d34-49f9-b8a1-6c08bf5efcc5" path="/var/lib/kubelet/pods/a4eec582-3d34-49f9-b8a1-6c08bf5efcc5/volumes"
Jul 07 17:45:17.200211 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:17.200173    2579 generic.go:356] "Generic (PLEG): container finished" podID="7768cc9c-a345-44e4-a564-3688a3c595e0" containerID="1fb05813f0cf3b08dce3e2730020a437e048327aa6d781a9a10817e1889d6c68" exitCode=0
Jul 07 17:45:17.200359 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:17.200212    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ce87c9/auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv" event={"ID":"7768cc9c-a345-44e4-a564-3688a3c595e0","Type":"ContainerDied","Data":"1fb05813f0cf3b08dce3e2730020a437e048327aa6d781a9a10817e1889d6c68"}
Jul 07 17:45:17.200359 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:17.200233    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ce87c9/auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv"
Jul 07 17:45:17.200359 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:17.200252    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ce87c9/auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv" event={"ID":"7768cc9c-a345-44e4-a564-3688a3c595e0","Type":"ContainerDied","Data":"b19d4208cfe67c907df5de8fd45ce69652ba541cbadd0d818414ddea97a1bd8d"}
Jul 07 17:45:17.200359 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:17.200271    2579 scope.go:117] "RemoveContainer" containerID="1fb05813f0cf3b08dce3e2730020a437e048327aa6d781a9a10817e1889d6c68"
Jul 07 17:45:17.208940 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:17.208923    2579 scope.go:117] "RemoveContainer" containerID="1fb05813f0cf3b08dce3e2730020a437e048327aa6d781a9a10817e1889d6c68"
Jul 07 17:45:17.209180 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:45:17.209159    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1fb05813f0cf3b08dce3e2730020a437e048327aa6d781a9a10817e1889d6c68\": container with ID starting with 1fb05813f0cf3b08dce3e2730020a437e048327aa6d781a9a10817e1889d6c68 not found: ID does not exist" containerID="1fb05813f0cf3b08dce3e2730020a437e048327aa6d781a9a10817e1889d6c68"
Jul 07 17:45:17.209281 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:17.209183    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fb05813f0cf3b08dce3e2730020a437e048327aa6d781a9a10817e1889d6c68"} err="failed to get container status \"1fb05813f0cf3b08dce3e2730020a437e048327aa6d781a9a10817e1889d6c68\": rpc error: code = NotFound desc = could not find container \"1fb05813f0cf3b08dce3e2730020a437e048327aa6d781a9a10817e1889d6c68\": container with ID starting with 1fb05813f0cf3b08dce3e2730020a437e048327aa6d781a9a10817e1889d6c68 not found: ID does not exist"
Jul 07 17:45:17.218788 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:17.218767    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ce87c9/auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv"]
Jul 07 17:45:17.223298 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:17.223277    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-ce87c9/auth-test-model-ce87c9-kserve-7999cb7c5f-k6ntv"]
Jul 07 17:45:19.028548 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:19.028477    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7768cc9c-a345-44e4-a564-3688a3c595e0" path="/var/lib/kubelet/pods/7768cc9c-a345-44e4-a564-3688a3c595e0/volumes"
Jul 07 17:45:20.603905 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:20.603840    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-19c9b1/sub-test-model-19c9b1-kserve-56787b8b45-rk96c"
Jul 07 17:45:20.604267 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:20.603931    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-19c9b1/sub-test-model-19c9b1-kserve-56787b8b45-rk96c"
Jul 07 17:45:20.616123 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:20.616097    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-19c9b1/sub-test-model-19c9b1-kserve-56787b8b45-rk96c"
Jul 07 17:45:21.231527 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:21.231494    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-19c9b1/sub-test-model-19c9b1-kserve-56787b8b45-rk96c"
Jul 07 17:45:29.387735 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:29.387706    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-ce87c9/sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf"]
Jul 07 17:45:29.388168 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:29.388064    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4eec582-3d34-49f9-b8a1-6c08bf5efcc5" containerName="main"
Jul 07 17:45:29.388168 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:29.388080    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4eec582-3d34-49f9-b8a1-6c08bf5efcc5" containerName="main"
Jul 07 17:45:29.388168 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:29.388099    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7768cc9c-a345-44e4-a564-3688a3c595e0" containerName="main"
Jul 07 17:45:29.388168 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:29.388105    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7768cc9c-a345-44e4-a564-3688a3c595e0" containerName="main"
Jul 07 17:45:29.388168 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:29.388162    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4eec582-3d34-49f9-b8a1-6c08bf5efcc5" containerName="main"
Jul 07 17:45:29.388168 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:29.388170    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7768cc9c-a345-44e4-a564-3688a3c595e0" containerName="main"
Jul 07 17:45:29.392409 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:29.392393    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ce87c9/sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf"
Jul 07 17:45:29.394702 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:29.394680    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ce87c9\"/\"kube-root-ca.crt\""
Jul 07 17:45:29.395421 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:29.395406    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ce87c9\"/\"openshift-service-ca.crt\""
Jul 07 17:45:29.395529 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:29.395442    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ce87c9\"/\"default-dockercfg-9qmxp\""
Jul 07 17:45:29.395596 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:29.395548    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ce87c9\"/\"sub-test-model-ce87c9-kserve-self-signed-certs\""
Jul 07 17:45:29.402848 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:29.402829    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ce87c9/sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf"]
Jul 07 17:45:29.550989 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:29.550957    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4535dfe5-300a-4ada-9c37-dbc807772c01-dshm\") pod \"sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf\" (UID: \"4535dfe5-300a-4ada-9c37-dbc807772c01\") " pod="ai-tenant-e2e-shared-b-ce87c9/sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf"
Jul 07 17:45:29.550989 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:29.550990    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vzk2\" (UniqueName: \"kubernetes.io/projected/4535dfe5-300a-4ada-9c37-dbc807772c01-kube-api-access-6vzk2\") pod \"sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf\" (UID: \"4535dfe5-300a-4ada-9c37-dbc807772c01\") " pod="ai-tenant-e2e-shared-b-ce87c9/sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf"
Jul 07 17:45:29.551208 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:29.551008    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4535dfe5-300a-4ada-9c37-dbc807772c01-home\") pod \"sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf\" (UID: \"4535dfe5-300a-4ada-9c37-dbc807772c01\") " pod="ai-tenant-e2e-shared-b-ce87c9/sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf"
Jul 07 17:45:29.551208 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:29.551047    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4535dfe5-300a-4ada-9c37-dbc807772c01-tls-certs\") pod \"sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf\" (UID: \"4535dfe5-300a-4ada-9c37-dbc807772c01\") " pod="ai-tenant-e2e-shared-b-ce87c9/sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf"
Jul 07 17:45:29.551208 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:29.551098    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4535dfe5-300a-4ada-9c37-dbc807772c01-tmp-dir\") pod \"sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf\" (UID: \"4535dfe5-300a-4ada-9c37-dbc807772c01\") " pod="ai-tenant-e2e-shared-b-ce87c9/sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf"
Jul 07 17:45:29.551208 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:29.551129    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4535dfe5-300a-4ada-9c37-dbc807772c01-model-cache\") pod \"sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf\" (UID: \"4535dfe5-300a-4ada-9c37-dbc807772c01\") " pod="ai-tenant-e2e-shared-b-ce87c9/sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf"
Jul 07 17:45:29.651751 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:29.651727    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4535dfe5-300a-4ada-9c37-dbc807772c01-tls-certs\") pod \"sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf\" (UID: \"4535dfe5-300a-4ada-9c37-dbc807772c01\") " pod="ai-tenant-e2e-shared-b-ce87c9/sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf"
Jul 07 17:45:29.651942 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:29.651763    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4535dfe5-300a-4ada-9c37-dbc807772c01-tmp-dir\") pod \"sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf\" (UID: \"4535dfe5-300a-4ada-9c37-dbc807772c01\") " pod="ai-tenant-e2e-shared-b-ce87c9/sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf"
Jul 07 17:45:29.651942 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:29.651793    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4535dfe5-300a-4ada-9c37-dbc807772c01-model-cache\") pod \"sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf\" (UID: \"4535dfe5-300a-4ada-9c37-dbc807772c01\") " pod="ai-tenant-e2e-shared-b-ce87c9/sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf"
Jul 07 17:45:29.651942 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:29.651844    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4535dfe5-300a-4ada-9c37-dbc807772c01-dshm\") pod \"sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf\" (UID: \"4535dfe5-300a-4ada-9c37-dbc807772c01\") " pod="ai-tenant-e2e-shared-b-ce87c9/sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf"
Jul 07 17:45:29.651942 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:29.651861    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vzk2\" (UniqueName: \"kubernetes.io/projected/4535dfe5-300a-4ada-9c37-dbc807772c01-kube-api-access-6vzk2\") pod \"sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf\" (UID: \"4535dfe5-300a-4ada-9c37-dbc807772c01\") " pod="ai-tenant-e2e-shared-b-ce87c9/sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf"
Jul 07 17:45:29.651942 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:29.651877    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4535dfe5-300a-4ada-9c37-dbc807772c01-home\") pod \"sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf\" (UID: \"4535dfe5-300a-4ada-9c37-dbc807772c01\") " pod="ai-tenant-e2e-shared-b-ce87c9/sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf"
Jul 07 17:45:29.652242 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:29.652221    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4535dfe5-300a-4ada-9c37-dbc807772c01-tmp-dir\") pod \"sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf\" (UID: \"4535dfe5-300a-4ada-9c37-dbc807772c01\") " pod="ai-tenant-e2e-shared-b-ce87c9/sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf"
Jul 07 17:45:29.652305 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:29.652284    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4535dfe5-300a-4ada-9c37-dbc807772c01-model-cache\") pod \"sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf\" (UID: \"4535dfe5-300a-4ada-9c37-dbc807772c01\") " pod="ai-tenant-e2e-shared-b-ce87c9/sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf"
Jul 07 17:45:29.652372 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:29.652352    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4535dfe5-300a-4ada-9c37-dbc807772c01-home\") pod \"sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf\" (UID: \"4535dfe5-300a-4ada-9c37-dbc807772c01\") " pod="ai-tenant-e2e-shared-b-ce87c9/sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf"
Jul 07 17:45:29.654238 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:29.654220    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4535dfe5-300a-4ada-9c37-dbc807772c01-dshm\") pod \"sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf\" (UID: \"4535dfe5-300a-4ada-9c37-dbc807772c01\") " pod="ai-tenant-e2e-shared-b-ce87c9/sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf"
Jul 07 17:45:29.654405 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:29.654389    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4535dfe5-300a-4ada-9c37-dbc807772c01-tls-certs\") pod \"sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf\" (UID: \"4535dfe5-300a-4ada-9c37-dbc807772c01\") " pod="ai-tenant-e2e-shared-b-ce87c9/sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf"
Jul 07 17:45:29.660952 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:29.660926    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vzk2\" (UniqueName: \"kubernetes.io/projected/4535dfe5-300a-4ada-9c37-dbc807772c01-kube-api-access-6vzk2\") pod \"sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf\" (UID: \"4535dfe5-300a-4ada-9c37-dbc807772c01\") " pod="ai-tenant-e2e-shared-b-ce87c9/sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf"
Jul 07 17:45:29.701642 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:29.701619    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ce87c9/sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf"
Jul 07 17:45:29.833038 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:29.833007    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ce87c9/sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf"]
Jul 07 17:45:29.834057 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:45:29.834031    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4535dfe5_300a_4ada_9c37_dbc807772c01.slice/crio-ab8c6d5ff25bd31af266df6f04bbd67aa49f9b89f8fb05f2afcf5bb4a3b80649 WatchSource:0}: Error finding container ab8c6d5ff25bd31af266df6f04bbd67aa49f9b89f8fb05f2afcf5bb4a3b80649: Status 404 returned error can't find the container with id ab8c6d5ff25bd31af266df6f04bbd67aa49f9b89f8fb05f2afcf5bb4a3b80649
Jul 07 17:45:30.270523 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:30.270484    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ce87c9/sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf" event={"ID":"4535dfe5-300a-4ada-9c37-dbc807772c01","Type":"ContainerStarted","Data":"345ceb2496722982d1eaf52010c7d70495f5cf5a301c28d2c0e4b11b4800700b"}
Jul 07 17:45:30.270692 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:30.270533    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ce87c9/sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf" event={"ID":"4535dfe5-300a-4ada-9c37-dbc807772c01","Type":"ContainerStarted","Data":"ab8c6d5ff25bd31af266df6f04bbd67aa49f9b89f8fb05f2afcf5bb4a3b80649"}
Jul 07 17:45:30.290932 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:30.290869    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-ce87c9/sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf" podStartSLOduration=1.2908551400000001 podStartE2EDuration="1.29085514s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:45:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:45:30.289380178 +0000 UTC m=+2253.785458685" watchObservedRunningTime="2026-07-07 17:45:30.29085514 +0000 UTC m=+2253.786933651"
Jul 07 17:45:39.701846 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:39.701813    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-ce87c9/sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf"
Jul 07 17:45:39.702246 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:39.701874    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-ce87c9/sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf"
Jul 07 17:45:39.713910 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:39.713864    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-ce87c9/sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf"
Jul 07 17:45:40.339137 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:45:40.339107    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-ce87c9/sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf"
Jul 07 17:46:20.803003 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:20.802971    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-19c9b1/sub-test-model-19c9b1-kserve-56787b8b45-rk96c"]
Jul 07 17:46:21.865070 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:21.865035    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ce87c9/sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf"]
Jul 07 17:46:33.393334 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:33.393302    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-19c9b1/rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc"]
Jul 07 17:46:33.396813 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:33.396796    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-19c9b1/rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc"
Jul 07 17:46:33.399715 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:33.399699    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-19c9b1\"/\"rate-test-model-19c9b1-kserve-self-signed-certs\""
Jul 07 17:46:33.410429 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:33.410410    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-19c9b1/rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc"]
Jul 07 17:46:33.583018 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:33.582987    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0cce40d4-d28c-49d0-a627-4cd730aa2e9f-tls-certs\") pod \"rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc\" (UID: \"0cce40d4-d28c-49d0-a627-4cd730aa2e9f\") " pod="ai-tenant-e2e-shared-a-19c9b1/rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc"
Jul 07 17:46:33.583018 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:33.583020    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h27mk\" (UniqueName: \"kubernetes.io/projected/0cce40d4-d28c-49d0-a627-4cd730aa2e9f-kube-api-access-h27mk\") pod \"rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc\" (UID: \"0cce40d4-d28c-49d0-a627-4cd730aa2e9f\") " pod="ai-tenant-e2e-shared-a-19c9b1/rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc"
Jul 07 17:46:33.583246 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:33.583047    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0cce40d4-d28c-49d0-a627-4cd730aa2e9f-tmp-dir\") pod \"rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc\" (UID: \"0cce40d4-d28c-49d0-a627-4cd730aa2e9f\") " pod="ai-tenant-e2e-shared-a-19c9b1/rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc"
Jul 07 17:46:33.583246 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:33.583138    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0cce40d4-d28c-49d0-a627-4cd730aa2e9f-model-cache\") pod \"rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc\" (UID: \"0cce40d4-d28c-49d0-a627-4cd730aa2e9f\") " pod="ai-tenant-e2e-shared-a-19c9b1/rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc"
Jul 07 17:46:33.583246 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:33.583184    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0cce40d4-d28c-49d0-a627-4cd730aa2e9f-dshm\") pod \"rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc\" (UID: \"0cce40d4-d28c-49d0-a627-4cd730aa2e9f\") " pod="ai-tenant-e2e-shared-a-19c9b1/rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc"
Jul 07 17:46:33.583246 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:33.583202    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0cce40d4-d28c-49d0-a627-4cd730aa2e9f-home\") pod \"rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc\" (UID: \"0cce40d4-d28c-49d0-a627-4cd730aa2e9f\") " pod="ai-tenant-e2e-shared-a-19c9b1/rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc"
Jul 07 17:46:33.683864 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:33.683779    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0cce40d4-d28c-49d0-a627-4cd730aa2e9f-model-cache\") pod \"rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc\" (UID: \"0cce40d4-d28c-49d0-a627-4cd730aa2e9f\") " pod="ai-tenant-e2e-shared-a-19c9b1/rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc"
Jul 07 17:46:33.683864 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:33.683818    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0cce40d4-d28c-49d0-a627-4cd730aa2e9f-dshm\") pod \"rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc\" (UID: \"0cce40d4-d28c-49d0-a627-4cd730aa2e9f\") " pod="ai-tenant-e2e-shared-a-19c9b1/rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc"
Jul 07 17:46:33.683864 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:33.683834    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0cce40d4-d28c-49d0-a627-4cd730aa2e9f-home\") pod \"rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc\" (UID: \"0cce40d4-d28c-49d0-a627-4cd730aa2e9f\") " pod="ai-tenant-e2e-shared-a-19c9b1/rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc"
Jul 07 17:46:33.684161 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:33.683873    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0cce40d4-d28c-49d0-a627-4cd730aa2e9f-tls-certs\") pod \"rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc\" (UID: \"0cce40d4-d28c-49d0-a627-4cd730aa2e9f\") " pod="ai-tenant-e2e-shared-a-19c9b1/rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc"
Jul 07 17:46:33.684161 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:33.683908    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h27mk\" (UniqueName: \"kubernetes.io/projected/0cce40d4-d28c-49d0-a627-4cd730aa2e9f-kube-api-access-h27mk\") pod \"rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc\" (UID: \"0cce40d4-d28c-49d0-a627-4cd730aa2e9f\") " pod="ai-tenant-e2e-shared-a-19c9b1/rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc"
Jul 07 17:46:33.684161 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:33.683932    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0cce40d4-d28c-49d0-a627-4cd730aa2e9f-tmp-dir\") pod \"rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc\" (UID: \"0cce40d4-d28c-49d0-a627-4cd730aa2e9f\") " pod="ai-tenant-e2e-shared-a-19c9b1/rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc"
Jul 07 17:46:33.684326 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:33.684236    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0cce40d4-d28c-49d0-a627-4cd730aa2e9f-model-cache\") pod \"rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc\" (UID: \"0cce40d4-d28c-49d0-a627-4cd730aa2e9f\") " pod="ai-tenant-e2e-shared-a-19c9b1/rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc"
Jul 07 17:46:33.684326 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:33.684290    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0cce40d4-d28c-49d0-a627-4cd730aa2e9f-home\") pod \"rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc\" (UID: \"0cce40d4-d28c-49d0-a627-4cd730aa2e9f\") " pod="ai-tenant-e2e-shared-a-19c9b1/rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc"
Jul 07 17:46:33.684418 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:33.684338    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0cce40d4-d28c-49d0-a627-4cd730aa2e9f-tmp-dir\") pod \"rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc\" (UID: \"0cce40d4-d28c-49d0-a627-4cd730aa2e9f\") " pod="ai-tenant-e2e-shared-a-19c9b1/rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc"
Jul 07 17:46:33.686213 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:33.686191    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0cce40d4-d28c-49d0-a627-4cd730aa2e9f-dshm\") pod \"rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc\" (UID: \"0cce40d4-d28c-49d0-a627-4cd730aa2e9f\") " pod="ai-tenant-e2e-shared-a-19c9b1/rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc"
Jul 07 17:46:33.686478 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:33.686460    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0cce40d4-d28c-49d0-a627-4cd730aa2e9f-tls-certs\") pod \"rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc\" (UID: \"0cce40d4-d28c-49d0-a627-4cd730aa2e9f\") " pod="ai-tenant-e2e-shared-a-19c9b1/rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc"
Jul 07 17:46:33.694064 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:33.694029    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h27mk\" (UniqueName: \"kubernetes.io/projected/0cce40d4-d28c-49d0-a627-4cd730aa2e9f-kube-api-access-h27mk\") pod \"rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc\" (UID: \"0cce40d4-d28c-49d0-a627-4cd730aa2e9f\") " pod="ai-tenant-e2e-shared-a-19c9b1/rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc"
Jul 07 17:46:33.706255 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:33.705949    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-19c9b1/rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc"
Jul 07 17:46:33.841540 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:33.841515    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-19c9b1/rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc"]
Jul 07 17:46:33.843268 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:46:33.843234    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0cce40d4_d28c_49d0_a627_4cd730aa2e9f.slice/crio-bce73100820108a639c9dbed164bd88ffa4a2a4c3b7e883a75715f11be66f664 WatchSource:0}: Error finding container bce73100820108a639c9dbed164bd88ffa4a2a4c3b7e883a75715f11be66f664: Status 404 returned error can't find the container with id bce73100820108a639c9dbed164bd88ffa4a2a4c3b7e883a75715f11be66f664
Jul 07 17:46:34.600929 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:34.600876    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-19c9b1/rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc" event={"ID":"0cce40d4-d28c-49d0-a627-4cd730aa2e9f","Type":"ContainerStarted","Data":"1589fd86d4ef5c67f45bb766420535e901bf1f3d36e2d6cb0c82fae809bf1c3a"}
Jul 07 17:46:34.600929 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:34.600930    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-19c9b1/rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc" event={"ID":"0cce40d4-d28c-49d0-a627-4cd730aa2e9f","Type":"ContainerStarted","Data":"bce73100820108a639c9dbed164bd88ffa4a2a4c3b7e883a75715f11be66f664"}
Jul 07 17:46:34.619322 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:34.619270    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-19c9b1/rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc" podStartSLOduration=1.6192556310000001 podStartE2EDuration="1.619255631s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:46:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:46:34.618559184 +0000 UTC m=+2318.114637695" watchObservedRunningTime="2026-07-07 17:46:34.619255631 +0000 UTC m=+2318.115334139"
Jul 07 17:46:35.823723 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:35.823667    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-19c9b1/sub-test-model-19c9b1-kserve-56787b8b45-rk96c" podUID="1449b9a5-4318-4df3-813d-4cfd35ae88a3" containerName="main" containerID="cri-o://e9a41cbe2bb2ca4276573b283676030e4f7a4b3c0b9eb836f351faad9f3d0ec6" gracePeriod=45
Jul 07 17:46:36.077154 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:36.077101    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-19c9b1/sub-test-model-19c9b1-kserve-56787b8b45-rk96c"
Jul 07 17:46:36.203835 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:36.203797    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1449b9a5-4318-4df3-813d-4cfd35ae88a3-home\") pod \"1449b9a5-4318-4df3-813d-4cfd35ae88a3\" (UID: \"1449b9a5-4318-4df3-813d-4cfd35ae88a3\") "
Jul 07 17:46:36.203835 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:36.203849    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1449b9a5-4318-4df3-813d-4cfd35ae88a3-dshm\") pod \"1449b9a5-4318-4df3-813d-4cfd35ae88a3\" (UID: \"1449b9a5-4318-4df3-813d-4cfd35ae88a3\") "
Jul 07 17:46:36.204088 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:36.203876    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1449b9a5-4318-4df3-813d-4cfd35ae88a3-model-cache\") pod \"1449b9a5-4318-4df3-813d-4cfd35ae88a3\" (UID: \"1449b9a5-4318-4df3-813d-4cfd35ae88a3\") "
Jul 07 17:46:36.204088 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:36.203944    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1449b9a5-4318-4df3-813d-4cfd35ae88a3-tls-certs\") pod \"1449b9a5-4318-4df3-813d-4cfd35ae88a3\" (UID: \"1449b9a5-4318-4df3-813d-4cfd35ae88a3\") "
Jul 07 17:46:36.204088 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:36.203967    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4btmp\" (UniqueName: \"kubernetes.io/projected/1449b9a5-4318-4df3-813d-4cfd35ae88a3-kube-api-access-4btmp\") pod \"1449b9a5-4318-4df3-813d-4cfd35ae88a3\" (UID: \"1449b9a5-4318-4df3-813d-4cfd35ae88a3\") "
Jul 07 17:46:36.204088 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:36.203983    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1449b9a5-4318-4df3-813d-4cfd35ae88a3-tmp-dir\") pod \"1449b9a5-4318-4df3-813d-4cfd35ae88a3\" (UID: \"1449b9a5-4318-4df3-813d-4cfd35ae88a3\") "
Jul 07 17:46:36.204088 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:36.204052    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1449b9a5-4318-4df3-813d-4cfd35ae88a3-home" (OuterVolumeSpecName: "home") pod "1449b9a5-4318-4df3-813d-4cfd35ae88a3" (UID: "1449b9a5-4318-4df3-813d-4cfd35ae88a3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:46:36.204354 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:36.204272    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1449b9a5-4318-4df3-813d-4cfd35ae88a3-home\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:46:36.204354 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:36.204270    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1449b9a5-4318-4df3-813d-4cfd35ae88a3-model-cache" (OuterVolumeSpecName: "model-cache") pod "1449b9a5-4318-4df3-813d-4cfd35ae88a3" (UID: "1449b9a5-4318-4df3-813d-4cfd35ae88a3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:46:36.204354 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:36.204329    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1449b9a5-4318-4df3-813d-4cfd35ae88a3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1449b9a5-4318-4df3-813d-4cfd35ae88a3" (UID: "1449b9a5-4318-4df3-813d-4cfd35ae88a3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:46:36.206018 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:36.205996    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1449b9a5-4318-4df3-813d-4cfd35ae88a3-dshm" (OuterVolumeSpecName: "dshm") pod "1449b9a5-4318-4df3-813d-4cfd35ae88a3" (UID: "1449b9a5-4318-4df3-813d-4cfd35ae88a3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:46:36.206127 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:36.206109    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1449b9a5-4318-4df3-813d-4cfd35ae88a3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1449b9a5-4318-4df3-813d-4cfd35ae88a3" (UID: "1449b9a5-4318-4df3-813d-4cfd35ae88a3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 17:46:36.206192 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:36.206159    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1449b9a5-4318-4df3-813d-4cfd35ae88a3-kube-api-access-4btmp" (OuterVolumeSpecName: "kube-api-access-4btmp") pod "1449b9a5-4318-4df3-813d-4cfd35ae88a3" (UID: "1449b9a5-4318-4df3-813d-4cfd35ae88a3"). InnerVolumeSpecName "kube-api-access-4btmp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 17:46:36.305428 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:36.305406    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1449b9a5-4318-4df3-813d-4cfd35ae88a3-tls-certs\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:46:36.305428 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:36.305427    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4btmp\" (UniqueName: \"kubernetes.io/projected/1449b9a5-4318-4df3-813d-4cfd35ae88a3-kube-api-access-4btmp\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:46:36.305561 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:36.305437    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1449b9a5-4318-4df3-813d-4cfd35ae88a3-tmp-dir\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:46:36.305561 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:36.305446    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1449b9a5-4318-4df3-813d-4cfd35ae88a3-dshm\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:46:36.305561 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:36.305453    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1449b9a5-4318-4df3-813d-4cfd35ae88a3-model-cache\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:46:36.612132 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:36.612097    2579 generic.go:356] "Generic (PLEG): container finished" podID="1449b9a5-4318-4df3-813d-4cfd35ae88a3" containerID="e9a41cbe2bb2ca4276573b283676030e4f7a4b3c0b9eb836f351faad9f3d0ec6" exitCode=0
Jul 07 17:46:36.612280 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:36.612138    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-19c9b1/sub-test-model-19c9b1-kserve-56787b8b45-rk96c"
Jul 07 17:46:36.612280 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:36.612156    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-19c9b1/sub-test-model-19c9b1-kserve-56787b8b45-rk96c" event={"ID":"1449b9a5-4318-4df3-813d-4cfd35ae88a3","Type":"ContainerDied","Data":"e9a41cbe2bb2ca4276573b283676030e4f7a4b3c0b9eb836f351faad9f3d0ec6"}
Jul 07 17:46:36.612280 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:36.612192    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-19c9b1/sub-test-model-19c9b1-kserve-56787b8b45-rk96c" event={"ID":"1449b9a5-4318-4df3-813d-4cfd35ae88a3","Type":"ContainerDied","Data":"1170056ac0080bf59926f1dc9129ce12c8fb7e0c80df92dba37d2a7553c16b89"}
Jul 07 17:46:36.612280 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:36.612215    2579 scope.go:117] "RemoveContainer" containerID="e9a41cbe2bb2ca4276573b283676030e4f7a4b3c0b9eb836f351faad9f3d0ec6"
Jul 07 17:46:36.621494 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:36.621474    2579 scope.go:117] "RemoveContainer" containerID="e9a41cbe2bb2ca4276573b283676030e4f7a4b3c0b9eb836f351faad9f3d0ec6"
Jul 07 17:46:36.621760 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:46:36.621742    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9a41cbe2bb2ca4276573b283676030e4f7a4b3c0b9eb836f351faad9f3d0ec6\": container with ID starting with e9a41cbe2bb2ca4276573b283676030e4f7a4b3c0b9eb836f351faad9f3d0ec6 not found: ID does not exist" containerID="e9a41cbe2bb2ca4276573b283676030e4f7a4b3c0b9eb836f351faad9f3d0ec6"
Jul 07 17:46:36.621809 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:36.621770    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9a41cbe2bb2ca4276573b283676030e4f7a4b3c0b9eb836f351faad9f3d0ec6"} err="failed to get container status \"e9a41cbe2bb2ca4276573b283676030e4f7a4b3c0b9eb836f351faad9f3d0ec6\": rpc error: code = NotFound desc = could not find container \"e9a41cbe2bb2ca4276573b283676030e4f7a4b3c0b9eb836f351faad9f3d0ec6\": container with ID starting with e9a41cbe2bb2ca4276573b283676030e4f7a4b3c0b9eb836f351faad9f3d0ec6 not found: ID does not exist"
Jul 07 17:46:36.634568 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:36.634545    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-19c9b1/sub-test-model-19c9b1-kserve-56787b8b45-rk96c"]
Jul 07 17:46:36.638987 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:36.638968    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-19c9b1/sub-test-model-19c9b1-kserve-56787b8b45-rk96c"]
Jul 07 17:46:36.884767 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:36.884685    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-ce87c9/sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf" podUID="4535dfe5-300a-4ada-9c37-dbc807772c01" containerName="main" containerID="cri-o://345ceb2496722982d1eaf52010c7d70495f5cf5a301c28d2c0e4b11b4800700b" gracePeriod=45
Jul 07 17:46:37.028831 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:37.028800    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1449b9a5-4318-4df3-813d-4cfd35ae88a3" path="/var/lib/kubelet/pods/1449b9a5-4318-4df3-813d-4cfd35ae88a3/volumes"
Jul 07 17:46:37.137543 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:37.137484    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ce87c9/sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf"
Jul 07 17:46:37.314098 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:37.314067    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4535dfe5-300a-4ada-9c37-dbc807772c01-model-cache\") pod \"4535dfe5-300a-4ada-9c37-dbc807772c01\" (UID: \"4535dfe5-300a-4ada-9c37-dbc807772c01\") "
Jul 07 17:46:37.314276 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:37.314110    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4535dfe5-300a-4ada-9c37-dbc807772c01-dshm\") pod \"4535dfe5-300a-4ada-9c37-dbc807772c01\" (UID: \"4535dfe5-300a-4ada-9c37-dbc807772c01\") "
Jul 07 17:46:37.314276 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:37.314132    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6vzk2\" (UniqueName: \"kubernetes.io/projected/4535dfe5-300a-4ada-9c37-dbc807772c01-kube-api-access-6vzk2\") pod \"4535dfe5-300a-4ada-9c37-dbc807772c01\" (UID: \"4535dfe5-300a-4ada-9c37-dbc807772c01\") "
Jul 07 17:46:37.314276 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:37.314154    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4535dfe5-300a-4ada-9c37-dbc807772c01-tls-certs\") pod \"4535dfe5-300a-4ada-9c37-dbc807772c01\" (UID: \"4535dfe5-300a-4ada-9c37-dbc807772c01\") "
Jul 07 17:46:37.314276 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:37.314195    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4535dfe5-300a-4ada-9c37-dbc807772c01-home\") pod \"4535dfe5-300a-4ada-9c37-dbc807772c01\" (UID: \"4535dfe5-300a-4ada-9c37-dbc807772c01\") "
Jul 07 17:46:37.314276 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:37.314243    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4535dfe5-300a-4ada-9c37-dbc807772c01-tmp-dir\") pod \"4535dfe5-300a-4ada-9c37-dbc807772c01\" (UID: \"4535dfe5-300a-4ada-9c37-dbc807772c01\") "
Jul 07 17:46:37.314538 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:37.314375    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4535dfe5-300a-4ada-9c37-dbc807772c01-model-cache" (OuterVolumeSpecName: "model-cache") pod "4535dfe5-300a-4ada-9c37-dbc807772c01" (UID: "4535dfe5-300a-4ada-9c37-dbc807772c01"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:46:37.314538 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:37.314499    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4535dfe5-300a-4ada-9c37-dbc807772c01-home" (OuterVolumeSpecName: "home") pod "4535dfe5-300a-4ada-9c37-dbc807772c01" (UID: "4535dfe5-300a-4ada-9c37-dbc807772c01"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:46:37.314538 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:37.314522    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4535dfe5-300a-4ada-9c37-dbc807772c01-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4535dfe5-300a-4ada-9c37-dbc807772c01" (UID: "4535dfe5-300a-4ada-9c37-dbc807772c01"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:46:37.314686 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:37.314584    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4535dfe5-300a-4ada-9c37-dbc807772c01-model-cache\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:46:37.314686 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:37.314603    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4535dfe5-300a-4ada-9c37-dbc807772c01-home\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:46:37.316350 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:37.316321    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4535dfe5-300a-4ada-9c37-dbc807772c01-dshm" (OuterVolumeSpecName: "dshm") pod "4535dfe5-300a-4ada-9c37-dbc807772c01" (UID: "4535dfe5-300a-4ada-9c37-dbc807772c01"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:46:37.316350 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:37.316340    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4535dfe5-300a-4ada-9c37-dbc807772c01-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4535dfe5-300a-4ada-9c37-dbc807772c01" (UID: "4535dfe5-300a-4ada-9c37-dbc807772c01"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 17:46:37.316499 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:37.316405    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4535dfe5-300a-4ada-9c37-dbc807772c01-kube-api-access-6vzk2" (OuterVolumeSpecName: "kube-api-access-6vzk2") pod "4535dfe5-300a-4ada-9c37-dbc807772c01" (UID: "4535dfe5-300a-4ada-9c37-dbc807772c01"). InnerVolumeSpecName "kube-api-access-6vzk2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 17:46:37.416040 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:37.416016    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4535dfe5-300a-4ada-9c37-dbc807772c01-dshm\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:46:37.416040 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:37.416038    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vzk2\" (UniqueName: \"kubernetes.io/projected/4535dfe5-300a-4ada-9c37-dbc807772c01-kube-api-access-6vzk2\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:46:37.416215 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:37.416050    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4535dfe5-300a-4ada-9c37-dbc807772c01-tls-certs\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:46:37.416215 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:37.416061    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4535dfe5-300a-4ada-9c37-dbc807772c01-tmp-dir\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:46:37.619215 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:37.619186    2579 generic.go:356] "Generic (PLEG): container finished" podID="4535dfe5-300a-4ada-9c37-dbc807772c01" containerID="345ceb2496722982d1eaf52010c7d70495f5cf5a301c28d2c0e4b11b4800700b" exitCode=0
Jul 07 17:46:37.619367 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:37.619221    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ce87c9/sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf"
Jul 07 17:46:37.619367 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:37.619239    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ce87c9/sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf" event={"ID":"4535dfe5-300a-4ada-9c37-dbc807772c01","Type":"ContainerDied","Data":"345ceb2496722982d1eaf52010c7d70495f5cf5a301c28d2c0e4b11b4800700b"}
Jul 07 17:46:37.619367 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:37.619270    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ce87c9/sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf" event={"ID":"4535dfe5-300a-4ada-9c37-dbc807772c01","Type":"ContainerDied","Data":"ab8c6d5ff25bd31af266df6f04bbd67aa49f9b89f8fb05f2afcf5bb4a3b80649"}
Jul 07 17:46:37.619367 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:37.619286    2579 scope.go:117] "RemoveContainer" containerID="345ceb2496722982d1eaf52010c7d70495f5cf5a301c28d2c0e4b11b4800700b"
Jul 07 17:46:37.628321 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:37.628305    2579 scope.go:117] "RemoveContainer" containerID="345ceb2496722982d1eaf52010c7d70495f5cf5a301c28d2c0e4b11b4800700b"
Jul 07 17:46:37.628572 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:46:37.628555    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"345ceb2496722982d1eaf52010c7d70495f5cf5a301c28d2c0e4b11b4800700b\": container with ID starting with 345ceb2496722982d1eaf52010c7d70495f5cf5a301c28d2c0e4b11b4800700b not found: ID does not exist" containerID="345ceb2496722982d1eaf52010c7d70495f5cf5a301c28d2c0e4b11b4800700b"
Jul 07 17:46:37.628640 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:37.628580    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"345ceb2496722982d1eaf52010c7d70495f5cf5a301c28d2c0e4b11b4800700b"} err="failed to get container status \"345ceb2496722982d1eaf52010c7d70495f5cf5a301c28d2c0e4b11b4800700b\": rpc error: code = NotFound desc = could not find container \"345ceb2496722982d1eaf52010c7d70495f5cf5a301c28d2c0e4b11b4800700b\": container with ID starting with 345ceb2496722982d1eaf52010c7d70495f5cf5a301c28d2c0e4b11b4800700b not found: ID does not exist"
Jul 07 17:46:37.643775 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:37.643744    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ce87c9/sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf"]
Jul 07 17:46:37.649160 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:37.649142    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-ce87c9/sub-test-model-ce87c9-kserve-59fbcc9c6f-926tf"]
Jul 07 17:46:39.027375 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:39.027343    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4535dfe5-300a-4ada-9c37-dbc807772c01" path="/var/lib/kubelet/pods/4535dfe5-300a-4ada-9c37-dbc807772c01/volumes"
Jul 07 17:46:43.706623 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:43.706586    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-19c9b1/rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc"
Jul 07 17:46:43.707058 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:43.706643    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-19c9b1/rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc"
Jul 07 17:46:43.718874 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:43.718849    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-19c9b1/rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc"
Jul 07 17:46:44.669647 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:44.669616    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-19c9b1/rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc"
Jul 07 17:46:58.110462 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:58.110429    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-ce87c9/rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn"]
Jul 07 17:46:58.110870 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:58.110753    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1449b9a5-4318-4df3-813d-4cfd35ae88a3" containerName="main"
Jul 07 17:46:58.110870 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:58.110764    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1449b9a5-4318-4df3-813d-4cfd35ae88a3" containerName="main"
Jul 07 17:46:58.110870 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:58.110778    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4535dfe5-300a-4ada-9c37-dbc807772c01" containerName="main"
Jul 07 17:46:58.110870 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:58.110783    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="4535dfe5-300a-4ada-9c37-dbc807772c01" containerName="main"
Jul 07 17:46:58.110870 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:58.110850    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="4535dfe5-300a-4ada-9c37-dbc807772c01" containerName="main"
Jul 07 17:46:58.110870 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:58.110860    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="1449b9a5-4318-4df3-813d-4cfd35ae88a3" containerName="main"
Jul 07 17:46:58.114783 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:58.114767    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ce87c9/rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn"
Jul 07 17:46:58.117114 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:58.117092    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ce87c9\"/\"openshift-service-ca.crt\""
Jul 07 17:46:58.117807 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:58.117789    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ce87c9\"/\"kube-root-ca.crt\""
Jul 07 17:46:58.117986 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:58.117965    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ce87c9\"/\"rate-test-model-ce87c9-kserve-self-signed-certs\""
Jul 07 17:46:58.118037 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:58.118018    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ce87c9\"/\"default-dockercfg-9qmxp\""
Jul 07 17:46:58.129406 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:58.129378    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ce87c9/rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn"]
Jul 07 17:46:58.184846 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:58.184811    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/791659eb-8ba7-43ac-9a08-ad5ff86f74ed-tmp-dir\") pod \"rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn\" (UID: \"791659eb-8ba7-43ac-9a08-ad5ff86f74ed\") " pod="ai-tenant-e2e-shared-b-ce87c9/rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn"
Jul 07 17:46:58.185028 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:58.184852    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/791659eb-8ba7-43ac-9a08-ad5ff86f74ed-dshm\") pod \"rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn\" (UID: \"791659eb-8ba7-43ac-9a08-ad5ff86f74ed\") " pod="ai-tenant-e2e-shared-b-ce87c9/rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn"
Jul 07 17:46:58.185028 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:58.184870    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/791659eb-8ba7-43ac-9a08-ad5ff86f74ed-model-cache\") pod \"rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn\" (UID: \"791659eb-8ba7-43ac-9a08-ad5ff86f74ed\") " pod="ai-tenant-e2e-shared-b-ce87c9/rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn"
Jul 07 17:46:58.185028 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:58.184910    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bc87z\" (UniqueName: \"kubernetes.io/projected/791659eb-8ba7-43ac-9a08-ad5ff86f74ed-kube-api-access-bc87z\") pod \"rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn\" (UID: \"791659eb-8ba7-43ac-9a08-ad5ff86f74ed\") " pod="ai-tenant-e2e-shared-b-ce87c9/rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn"
Jul 07 17:46:58.185028 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:58.184991    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/791659eb-8ba7-43ac-9a08-ad5ff86f74ed-tls-certs\") pod \"rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn\" (UID: \"791659eb-8ba7-43ac-9a08-ad5ff86f74ed\") " pod="ai-tenant-e2e-shared-b-ce87c9/rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn"
Jul 07 17:46:58.185232 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:58.185081    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/791659eb-8ba7-43ac-9a08-ad5ff86f74ed-home\") pod \"rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn\" (UID: \"791659eb-8ba7-43ac-9a08-ad5ff86f74ed\") " pod="ai-tenant-e2e-shared-b-ce87c9/rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn"
Jul 07 17:46:58.286347 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:58.286314    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/791659eb-8ba7-43ac-9a08-ad5ff86f74ed-home\") pod \"rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn\" (UID: \"791659eb-8ba7-43ac-9a08-ad5ff86f74ed\") " pod="ai-tenant-e2e-shared-b-ce87c9/rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn"
Jul 07 17:46:58.286548 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:58.286378    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/791659eb-8ba7-43ac-9a08-ad5ff86f74ed-tmp-dir\") pod \"rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn\" (UID: \"791659eb-8ba7-43ac-9a08-ad5ff86f74ed\") " pod="ai-tenant-e2e-shared-b-ce87c9/rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn"
Jul 07 17:46:58.286548 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:58.286408    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/791659eb-8ba7-43ac-9a08-ad5ff86f74ed-dshm\") pod \"rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn\" (UID: \"791659eb-8ba7-43ac-9a08-ad5ff86f74ed\") " pod="ai-tenant-e2e-shared-b-ce87c9/rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn"
Jul 07 17:46:58.286548 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:58.286443    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/791659eb-8ba7-43ac-9a08-ad5ff86f74ed-model-cache\") pod \"rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn\" (UID: \"791659eb-8ba7-43ac-9a08-ad5ff86f74ed\") " pod="ai-tenant-e2e-shared-b-ce87c9/rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn"
Jul 07 17:46:58.286548 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:58.286462    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bc87z\" (UniqueName: \"kubernetes.io/projected/791659eb-8ba7-43ac-9a08-ad5ff86f74ed-kube-api-access-bc87z\") pod \"rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn\" (UID: \"791659eb-8ba7-43ac-9a08-ad5ff86f74ed\") " pod="ai-tenant-e2e-shared-b-ce87c9/rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn"
Jul 07 17:46:58.286776 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:58.286611    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/791659eb-8ba7-43ac-9a08-ad5ff86f74ed-tls-certs\") pod \"rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn\" (UID: \"791659eb-8ba7-43ac-9a08-ad5ff86f74ed\") " pod="ai-tenant-e2e-shared-b-ce87c9/rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn"
Jul 07 17:46:58.286866 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:58.286843    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/791659eb-8ba7-43ac-9a08-ad5ff86f74ed-home\") pod \"rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn\" (UID: \"791659eb-8ba7-43ac-9a08-ad5ff86f74ed\") " pod="ai-tenant-e2e-shared-b-ce87c9/rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn"
Jul 07 17:46:58.286963 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:58.286921    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/791659eb-8ba7-43ac-9a08-ad5ff86f74ed-tmp-dir\") pod \"rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn\" (UID: \"791659eb-8ba7-43ac-9a08-ad5ff86f74ed\") " pod="ai-tenant-e2e-shared-b-ce87c9/rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn"
Jul 07 17:46:58.287003 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:58.286982    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/791659eb-8ba7-43ac-9a08-ad5ff86f74ed-model-cache\") pod \"rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn\" (UID: \"791659eb-8ba7-43ac-9a08-ad5ff86f74ed\") " pod="ai-tenant-e2e-shared-b-ce87c9/rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn"
Jul 07 17:46:58.288800 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:58.288781    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/791659eb-8ba7-43ac-9a08-ad5ff86f74ed-dshm\") pod \"rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn\" (UID: \"791659eb-8ba7-43ac-9a08-ad5ff86f74ed\") " pod="ai-tenant-e2e-shared-b-ce87c9/rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn"
Jul 07 17:46:58.289117 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:58.289097    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/791659eb-8ba7-43ac-9a08-ad5ff86f74ed-tls-certs\") pod \"rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn\" (UID: \"791659eb-8ba7-43ac-9a08-ad5ff86f74ed\") " pod="ai-tenant-e2e-shared-b-ce87c9/rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn"
Jul 07 17:46:58.295779 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:58.295757    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bc87z\" (UniqueName: \"kubernetes.io/projected/791659eb-8ba7-43ac-9a08-ad5ff86f74ed-kube-api-access-bc87z\") pod \"rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn\" (UID: \"791659eb-8ba7-43ac-9a08-ad5ff86f74ed\") " pod="ai-tenant-e2e-shared-b-ce87c9/rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn"
Jul 07 17:46:58.423993 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:58.423955    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ce87c9/rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn"
Jul 07 17:46:58.562342 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:58.562317    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ce87c9/rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn"]
Jul 07 17:46:58.564440 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:46:58.564414    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod791659eb_8ba7_43ac_9a08_ad5ff86f74ed.slice/crio-9624540ac18432859737d4ea79b7280e93c2b6b774ae2bbaf41e2b1cca08fd70 WatchSource:0}: Error finding container 9624540ac18432859737d4ea79b7280e93c2b6b774ae2bbaf41e2b1cca08fd70: Status 404 returned error can't find the container with id 9624540ac18432859737d4ea79b7280e93c2b6b774ae2bbaf41e2b1cca08fd70
Jul 07 17:46:58.733763 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:58.733661    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ce87c9/rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn" event={"ID":"791659eb-8ba7-43ac-9a08-ad5ff86f74ed","Type":"ContainerStarted","Data":"b3785b17eafcd0518b03c9c9d8fb6f3dc93c8a3a354755a655d8ed837c47f4e8"}
Jul 07 17:46:58.733763 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:58.733712    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ce87c9/rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn" event={"ID":"791659eb-8ba7-43ac-9a08-ad5ff86f74ed","Type":"ContainerStarted","Data":"9624540ac18432859737d4ea79b7280e93c2b6b774ae2bbaf41e2b1cca08fd70"}
Jul 07 17:46:58.754972 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:46:58.754912    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-ce87c9/rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn" podStartSLOduration=0.754883452 podStartE2EDuration="754.883452ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 17:46:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:46:58.752290865 +0000 UTC m=+2342.248369387" watchObservedRunningTime="2026-07-07 17:46:58.754883452 +0000 UTC m=+2342.250974040"
Jul 07 17:47:08.424221 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:47:08.424186    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-ce87c9/rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn"
Jul 07 17:47:08.424705 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:47:08.424235    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-ce87c9/rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn"
Jul 07 17:47:08.436269 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:47:08.436242    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-ce87c9/rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn"
Jul 07 17:47:08.794255 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:47:08.794178    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-ce87c9/rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn"
Jul 07 17:47:44.972823 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:47:44.972791    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-19c9b1/rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc"]
Jul 07 17:47:45.798410 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:47:45.798373    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ce87c9/rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn"]
Jul 07 17:47:59.993266 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:47:59.993227    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-19c9b1/rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc" podUID="0cce40d4-d28c-49d0-a627-4cd730aa2e9f" containerName="main" containerID="cri-o://1589fd86d4ef5c67f45bb766420535e901bf1f3d36e2d6cb0c82fae809bf1c3a" gracePeriod=45
Jul 07 17:48:00.237958 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:00.237938    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-19c9b1/rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc"
Jul 07 17:48:00.291428 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:00.291370    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0cce40d4-d28c-49d0-a627-4cd730aa2e9f-home\") pod \"0cce40d4-d28c-49d0-a627-4cd730aa2e9f\" (UID: \"0cce40d4-d28c-49d0-a627-4cd730aa2e9f\") "
Jul 07 17:48:00.291428 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:00.291407    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0cce40d4-d28c-49d0-a627-4cd730aa2e9f-model-cache\") pod \"0cce40d4-d28c-49d0-a627-4cd730aa2e9f\" (UID: \"0cce40d4-d28c-49d0-a627-4cd730aa2e9f\") "
Jul 07 17:48:00.291606 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:00.291435    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0cce40d4-d28c-49d0-a627-4cd730aa2e9f-dshm\") pod \"0cce40d4-d28c-49d0-a627-4cd730aa2e9f\" (UID: \"0cce40d4-d28c-49d0-a627-4cd730aa2e9f\") "
Jul 07 17:48:00.291606 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:00.291459    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0cce40d4-d28c-49d0-a627-4cd730aa2e9f-tls-certs\") pod \"0cce40d4-d28c-49d0-a627-4cd730aa2e9f\" (UID: \"0cce40d4-d28c-49d0-a627-4cd730aa2e9f\") "
Jul 07 17:48:00.291606 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:00.291477    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0cce40d4-d28c-49d0-a627-4cd730aa2e9f-tmp-dir\") pod \"0cce40d4-d28c-49d0-a627-4cd730aa2e9f\" (UID: \"0cce40d4-d28c-49d0-a627-4cd730aa2e9f\") "
Jul 07 17:48:00.291606 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:00.291499    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h27mk\" (UniqueName: \"kubernetes.io/projected/0cce40d4-d28c-49d0-a627-4cd730aa2e9f-kube-api-access-h27mk\") pod \"0cce40d4-d28c-49d0-a627-4cd730aa2e9f\" (UID: \"0cce40d4-d28c-49d0-a627-4cd730aa2e9f\") "
Jul 07 17:48:00.291819 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:00.291637    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cce40d4-d28c-49d0-a627-4cd730aa2e9f-home" (OuterVolumeSpecName: "home") pod "0cce40d4-d28c-49d0-a627-4cd730aa2e9f" (UID: "0cce40d4-d28c-49d0-a627-4cd730aa2e9f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:48:00.291819 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:00.291657    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cce40d4-d28c-49d0-a627-4cd730aa2e9f-model-cache" (OuterVolumeSpecName: "model-cache") pod "0cce40d4-d28c-49d0-a627-4cd730aa2e9f" (UID: "0cce40d4-d28c-49d0-a627-4cd730aa2e9f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:48:00.291819 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:00.291745    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cce40d4-d28c-49d0-a627-4cd730aa2e9f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0cce40d4-d28c-49d0-a627-4cd730aa2e9f" (UID: "0cce40d4-d28c-49d0-a627-4cd730aa2e9f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:48:00.291819 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:00.291806    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0cce40d4-d28c-49d0-a627-4cd730aa2e9f-home\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:48:00.291819 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:00.291819    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0cce40d4-d28c-49d0-a627-4cd730aa2e9f-model-cache\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:48:00.293664 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:00.293641    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cce40d4-d28c-49d0-a627-4cd730aa2e9f-dshm" (OuterVolumeSpecName: "dshm") pod "0cce40d4-d28c-49d0-a627-4cd730aa2e9f" (UID: "0cce40d4-d28c-49d0-a627-4cd730aa2e9f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:48:00.293739 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:00.293681    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0cce40d4-d28c-49d0-a627-4cd730aa2e9f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0cce40d4-d28c-49d0-a627-4cd730aa2e9f" (UID: "0cce40d4-d28c-49d0-a627-4cd730aa2e9f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 17:48:00.293779 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:00.293729    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cce40d4-d28c-49d0-a627-4cd730aa2e9f-kube-api-access-h27mk" (OuterVolumeSpecName: "kube-api-access-h27mk") pod "0cce40d4-d28c-49d0-a627-4cd730aa2e9f" (UID: "0cce40d4-d28c-49d0-a627-4cd730aa2e9f"). InnerVolumeSpecName "kube-api-access-h27mk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 17:48:00.392939 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:00.392915    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0cce40d4-d28c-49d0-a627-4cd730aa2e9f-dshm\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:48:00.392939 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:00.392937    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0cce40d4-d28c-49d0-a627-4cd730aa2e9f-tls-certs\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:48:00.393068 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:00.392950    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0cce40d4-d28c-49d0-a627-4cd730aa2e9f-tmp-dir\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:48:00.393068 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:00.392961    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h27mk\" (UniqueName: \"kubernetes.io/projected/0cce40d4-d28c-49d0-a627-4cd730aa2e9f-kube-api-access-h27mk\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:48:00.818479 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:00.818431    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-ce87c9/rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn" podUID="791659eb-8ba7-43ac-9a08-ad5ff86f74ed" containerName="main" containerID="cri-o://b3785b17eafcd0518b03c9c9d8fb6f3dc93c8a3a354755a655d8ed837c47f4e8" gracePeriod=45
Jul 07 17:48:01.058627 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:01.058591    2579 generic.go:356] "Generic (PLEG): container finished" podID="0cce40d4-d28c-49d0-a627-4cd730aa2e9f" containerID="1589fd86d4ef5c67f45bb766420535e901bf1f3d36e2d6cb0c82fae809bf1c3a" exitCode=0
Jul 07 17:48:01.059029 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:01.058638    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-19c9b1/rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc"
Jul 07 17:48:01.059029 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:01.058666    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-19c9b1/rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc" event={"ID":"0cce40d4-d28c-49d0-a627-4cd730aa2e9f","Type":"ContainerDied","Data":"1589fd86d4ef5c67f45bb766420535e901bf1f3d36e2d6cb0c82fae809bf1c3a"}
Jul 07 17:48:01.059029 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:01.058729    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-19c9b1/rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc" event={"ID":"0cce40d4-d28c-49d0-a627-4cd730aa2e9f","Type":"ContainerDied","Data":"bce73100820108a639c9dbed164bd88ffa4a2a4c3b7e883a75715f11be66f664"}
Jul 07 17:48:01.059029 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:01.058743    2579 scope.go:117] "RemoveContainer" containerID="1589fd86d4ef5c67f45bb766420535e901bf1f3d36e2d6cb0c82fae809bf1c3a"
Jul 07 17:48:01.060405 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:01.060372    2579 generic.go:356] "Generic (PLEG): container finished" podID="791659eb-8ba7-43ac-9a08-ad5ff86f74ed" containerID="b3785b17eafcd0518b03c9c9d8fb6f3dc93c8a3a354755a655d8ed837c47f4e8" exitCode=0
Jul 07 17:48:01.060405 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:01.060402    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ce87c9/rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn" event={"ID":"791659eb-8ba7-43ac-9a08-ad5ff86f74ed","Type":"ContainerDied","Data":"b3785b17eafcd0518b03c9c9d8fb6f3dc93c8a3a354755a655d8ed837c47f4e8"}
Jul 07 17:48:01.060550 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:01.060443    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ce87c9/rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn" event={"ID":"791659eb-8ba7-43ac-9a08-ad5ff86f74ed","Type":"ContainerDied","Data":"9624540ac18432859737d4ea79b7280e93c2b6b774ae2bbaf41e2b1cca08fd70"}
Jul 07 17:48:01.060550 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:01.060460    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9624540ac18432859737d4ea79b7280e93c2b6b774ae2bbaf41e2b1cca08fd70"
Jul 07 17:48:01.069481 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:01.069435    2579 scope.go:117] "RemoveContainer" containerID="1589fd86d4ef5c67f45bb766420535e901bf1f3d36e2d6cb0c82fae809bf1c3a"
Jul 07 17:48:01.069676 ip-10-0-142-103 kubenswrapper[2579]: E0707 17:48:01.069659    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1589fd86d4ef5c67f45bb766420535e901bf1f3d36e2d6cb0c82fae809bf1c3a\": container with ID starting with 1589fd86d4ef5c67f45bb766420535e901bf1f3d36e2d6cb0c82fae809bf1c3a not found: ID does not exist" containerID="1589fd86d4ef5c67f45bb766420535e901bf1f3d36e2d6cb0c82fae809bf1c3a"
Jul 07 17:48:01.069747 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:01.069684    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1589fd86d4ef5c67f45bb766420535e901bf1f3d36e2d6cb0c82fae809bf1c3a"} err="failed to get container status \"1589fd86d4ef5c67f45bb766420535e901bf1f3d36e2d6cb0c82fae809bf1c3a\": rpc error: code = NotFound desc = could not find container \"1589fd86d4ef5c67f45bb766420535e901bf1f3d36e2d6cb0c82fae809bf1c3a\": container with ID starting with 1589fd86d4ef5c67f45bb766420535e901bf1f3d36e2d6cb0c82fae809bf1c3a not found: ID does not exist"
Jul 07 17:48:01.073036 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:01.073020    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ce87c9/rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn"
Jul 07 17:48:01.089268 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:01.089243    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-19c9b1/rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc"]
Jul 07 17:48:01.093228 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:01.093210    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-19c9b1/rate-test-model-19c9b1-kserve-68cc8d6855-pmfzc"]
Jul 07 17:48:01.198253 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:01.198230    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/791659eb-8ba7-43ac-9a08-ad5ff86f74ed-home\") pod \"791659eb-8ba7-43ac-9a08-ad5ff86f74ed\" (UID: \"791659eb-8ba7-43ac-9a08-ad5ff86f74ed\") "
Jul 07 17:48:01.198359 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:01.198269    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/791659eb-8ba7-43ac-9a08-ad5ff86f74ed-tls-certs\") pod \"791659eb-8ba7-43ac-9a08-ad5ff86f74ed\" (UID: \"791659eb-8ba7-43ac-9a08-ad5ff86f74ed\") "
Jul 07 17:48:01.198359 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:01.198333    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/791659eb-8ba7-43ac-9a08-ad5ff86f74ed-dshm\") pod \"791659eb-8ba7-43ac-9a08-ad5ff86f74ed\" (UID: \"791659eb-8ba7-43ac-9a08-ad5ff86f74ed\") "
Jul 07 17:48:01.198446 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:01.198376    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/791659eb-8ba7-43ac-9a08-ad5ff86f74ed-model-cache\") pod \"791659eb-8ba7-43ac-9a08-ad5ff86f74ed\" (UID: \"791659eb-8ba7-43ac-9a08-ad5ff86f74ed\") "
Jul 07 17:48:01.198446 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:01.198405    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/791659eb-8ba7-43ac-9a08-ad5ff86f74ed-tmp-dir\") pod \"791659eb-8ba7-43ac-9a08-ad5ff86f74ed\" (UID: \"791659eb-8ba7-43ac-9a08-ad5ff86f74ed\") "
Jul 07 17:48:01.198446 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:01.198433    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bc87z\" (UniqueName: \"kubernetes.io/projected/791659eb-8ba7-43ac-9a08-ad5ff86f74ed-kube-api-access-bc87z\") pod \"791659eb-8ba7-43ac-9a08-ad5ff86f74ed\" (UID: \"791659eb-8ba7-43ac-9a08-ad5ff86f74ed\") "
Jul 07 17:48:01.198598 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:01.198521    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/791659eb-8ba7-43ac-9a08-ad5ff86f74ed-home" (OuterVolumeSpecName: "home") pod "791659eb-8ba7-43ac-9a08-ad5ff86f74ed" (UID: "791659eb-8ba7-43ac-9a08-ad5ff86f74ed"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:48:01.198711 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:01.198683    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/791659eb-8ba7-43ac-9a08-ad5ff86f74ed-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "791659eb-8ba7-43ac-9a08-ad5ff86f74ed" (UID: "791659eb-8ba7-43ac-9a08-ad5ff86f74ed"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:48:01.198778 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:01.198701    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/791659eb-8ba7-43ac-9a08-ad5ff86f74ed-model-cache" (OuterVolumeSpecName: "model-cache") pod "791659eb-8ba7-43ac-9a08-ad5ff86f74ed" (UID: "791659eb-8ba7-43ac-9a08-ad5ff86f74ed"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:48:01.198778 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:01.198694    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/791659eb-8ba7-43ac-9a08-ad5ff86f74ed-home\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:48:01.200508 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:01.200488    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/791659eb-8ba7-43ac-9a08-ad5ff86f74ed-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "791659eb-8ba7-43ac-9a08-ad5ff86f74ed" (UID: "791659eb-8ba7-43ac-9a08-ad5ff86f74ed"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 17:48:01.200584 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:01.200540    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/791659eb-8ba7-43ac-9a08-ad5ff86f74ed-dshm" (OuterVolumeSpecName: "dshm") pod "791659eb-8ba7-43ac-9a08-ad5ff86f74ed" (UID: "791659eb-8ba7-43ac-9a08-ad5ff86f74ed"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 17:48:01.200584 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:01.200549    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/791659eb-8ba7-43ac-9a08-ad5ff86f74ed-kube-api-access-bc87z" (OuterVolumeSpecName: "kube-api-access-bc87z") pod "791659eb-8ba7-43ac-9a08-ad5ff86f74ed" (UID: "791659eb-8ba7-43ac-9a08-ad5ff86f74ed"). InnerVolumeSpecName "kube-api-access-bc87z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 17:48:01.299906 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:01.299870    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/791659eb-8ba7-43ac-9a08-ad5ff86f74ed-dshm\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:48:01.299994 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:01.299910    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/791659eb-8ba7-43ac-9a08-ad5ff86f74ed-model-cache\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:48:01.299994 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:01.299921    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/791659eb-8ba7-43ac-9a08-ad5ff86f74ed-tmp-dir\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:48:01.299994 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:01.299929    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bc87z\" (UniqueName: \"kubernetes.io/projected/791659eb-8ba7-43ac-9a08-ad5ff86f74ed-kube-api-access-bc87z\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:48:01.299994 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:01.299937    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/791659eb-8ba7-43ac-9a08-ad5ff86f74ed-tls-certs\") on node \"ip-10-0-142-103.ec2.internal\" DevicePath \"\""
Jul 07 17:48:02.066044 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:02.066018    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ce87c9/rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn"
Jul 07 17:48:02.090128 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:02.090106    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ce87c9/rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn"]
Jul 07 17:48:02.099302 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:02.099282    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-ce87c9/rate-test-model-ce87c9-kserve-57cdcb756c-hqtbn"]
Jul 07 17:48:03.027388 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:03.027354    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0cce40d4-d28c-49d0-a627-4cd730aa2e9f" path="/var/lib/kubelet/pods/0cce40d4-d28c-49d0-a627-4cd730aa2e9f/volumes"
Jul 07 17:48:03.027732 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:03.027718    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="791659eb-8ba7-43ac-9a08-ad5ff86f74ed" path="/var/lib/kubelet/pods/791659eb-8ba7-43ac-9a08-ad5ff86f74ed/volumes"
Jul 07 17:48:22.253725 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:22.253620    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5c4697d5b4-9rpkm_4724ce74-14df-4bf5-a303-ba98f71b6ebe/authorino/0.log"
Jul 07 17:48:28.015122 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:28.015099    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-4nbnm_20330f0d-cf1b-489e-a8c7-866bfbd84e4f/server/0.log"
Jul 07 17:48:28.140489 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:28.140459    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-clcxv_cedceb3f-624f-4148-b0cb-529b9d1888d6/manager/0.log"
Jul 07 17:48:28.533418 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:28.533394    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-547959ffc4-rs7zv_6995de0d-adeb-44ea-8b4c-52d7464731bd/manager/0.log"
Jul 07 17:48:28.546498 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:28.546475    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-547959ffc4-rs7zv_6995de0d-adeb-44ea-8b4c-52d7464731bd/copy-manifests/0.log"
Jul 07 17:48:29.527767 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:29.527742    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75985h85_2bc00da5-eced-45fb-ac31-62874e629de7/util/0.log"
Jul 07 17:48:29.536673 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:29.536656    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75985h85_2bc00da5-eced-45fb-ac31-62874e629de7/pull/0.log"
Jul 07 17:48:29.545164 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:29.545134    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75985h85_2bc00da5-eced-45fb-ac31-62874e629de7/extract/0.log"
Jul 07 17:48:29.662707 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:29.662685    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04npwd_ff5c3733-6971-4414-9c3d-87d9676bea7c/util/0.log"
Jul 07 17:48:29.673710 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:29.673687    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04npwd_ff5c3733-6971-4414-9c3d-87d9676bea7c/pull/0.log"
Jul 07 17:48:29.685511 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:29.685493    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04npwd_ff5c3733-6971-4414-9c3d-87d9676bea7c/extract/0.log"
Jul 07 17:48:29.817679 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:29.817623    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvk57_f0112e35-a82a-43c2-9d3f-02613ca5e3e4/util/0.log"
Jul 07 17:48:29.825141 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:29.825122    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvk57_f0112e35-a82a-43c2-9d3f-02613ca5e3e4/pull/0.log"
Jul 07 17:48:29.832713 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:29.832698    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvk57_f0112e35-a82a-43c2-9d3f-02613ca5e3e4/extract/0.log"
Jul 07 17:48:29.952336 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:29.952316    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef199k25_218a3e4d-650c-4a0c-bd60-1670cffe7474/util/0.log"
Jul 07 17:48:29.960586 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:29.960568    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef199k25_218a3e4d-650c-4a0c-bd60-1670cffe7474/pull/0.log"
Jul 07 17:48:29.968241 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:29.968209    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef199k25_218a3e4d-650c-4a0c-bd60-1670cffe7474/extract/0.log"
Jul 07 17:48:30.113636 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:30.113573    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5c4697d5b4-9rpkm_4724ce74-14df-4bf5-a303-ba98f71b6ebe/authorino/0.log"
Jul 07 17:48:30.503150 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:30.503131    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-4cjgg_7e45d93a-d792-4ba2-996a-c8ac2f28c187/kuadrant-console-plugin/0.log"
Jul 07 17:48:31.779853 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:31.779819    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-76676f7fb7-ghktc_c201225c-e095-4494-b013-694764cd0272/kube-auth-proxy/0.log"
Jul 07 17:48:32.812733 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:32.812700    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-ftlqk_8c0c92c6-9968-43b2-97d9-e26e36415dbd/main/0.log"
Jul 07 17:48:33.072978 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:33.072906    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbvm6bt_268f092d-b8cd-4aa9-8c79-e28b19312630/main/0.log"
Jul 07 17:48:33.338347 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:33.338290    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-75lgx_90a24b15-407c-43f6-9d33-d3ab01909153/main/0.log"
Jul 07 17:48:41.204701 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:41.204668    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-wqh28_5d706921-ddd7-4b39-840d-a03cef437d64/global-pull-secret-syncer/0.log"
Jul 07 17:48:41.340557 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:41.340536    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-ktm2t_fc4d0910-9984-483e-9079-22b47c0defdb/konnectivity-agent/0.log"
Jul 07 17:48:41.419107 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:41.419084    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-103.ec2.internal_3df4ea321bf474671e41e2422091c862/haproxy/0.log"
Jul 07 17:48:45.390039 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:45.390010    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75985h85_2bc00da5-eced-45fb-ac31-62874e629de7/extract/0.log"
Jul 07 17:48:45.427609 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:45.427584    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75985h85_2bc00da5-eced-45fb-ac31-62874e629de7/util/0.log"
Jul 07 17:48:45.482900 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:45.482867    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75985h85_2bc00da5-eced-45fb-ac31-62874e629de7/pull/0.log"
Jul 07 17:48:45.555368 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:45.555347    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04npwd_ff5c3733-6971-4414-9c3d-87d9676bea7c/extract/0.log"
Jul 07 17:48:45.599070 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:45.599049    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04npwd_ff5c3733-6971-4414-9c3d-87d9676bea7c/util/0.log"
Jul 07 17:48:45.663525 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:45.663506    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04npwd_ff5c3733-6971-4414-9c3d-87d9676bea7c/pull/0.log"
Jul 07 17:48:45.724297 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:45.724279    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvk57_f0112e35-a82a-43c2-9d3f-02613ca5e3e4/extract/0.log"
Jul 07 17:48:45.762767 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:45.762745    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvk57_f0112e35-a82a-43c2-9d3f-02613ca5e3e4/util/0.log"
Jul 07 17:48:45.800936 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:45.800916    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jvk57_f0112e35-a82a-43c2-9d3f-02613ca5e3e4/pull/0.log"
Jul 07 17:48:45.859314 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:45.859294    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef199k25_218a3e4d-650c-4a0c-bd60-1670cffe7474/extract/0.log"
Jul 07 17:48:45.892791 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:45.892772    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef199k25_218a3e4d-650c-4a0c-bd60-1670cffe7474/util/0.log"
Jul 07 17:48:45.922857 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:45.922818    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef199k25_218a3e4d-650c-4a0c-bd60-1670cffe7474/pull/0.log"
Jul 07 17:48:46.185107 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:46.185042    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5c4697d5b4-9rpkm_4724ce74-14df-4bf5-a303-ba98f71b6ebe/authorino/0.log"
Jul 07 17:48:46.294355 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:46.294332    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-4cjgg_7e45d93a-d792-4ba2-996a-c8ac2f28c187/kuadrant-console-plugin/0.log"
Jul 07 17:48:48.373968 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:48.373876    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61/alertmanager/0.log"
Jul 07 17:48:48.417507 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:48.417447    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61/config-reloader/0.log"
Jul 07 17:48:48.455351 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:48.455324    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61/kube-rbac-proxy-web/0.log"
Jul 07 17:48:48.481127 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:48.481107    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61/kube-rbac-proxy/0.log"
Jul 07 17:48:48.510293 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:48.510277    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61/kube-rbac-proxy-metric/0.log"
Jul 07 17:48:48.536263 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:48.536245    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61/prom-label-proxy/0.log"
Jul 07 17:48:48.564033 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:48.564016    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1ec998c7-6f2c-4961-8c48-6ad5dcf5bd61/init-config-reloader/0.log"
Jul 07 17:48:48.779061 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:48.779038    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-gphrw_b5856597-5bb9-4e7d-ab73-85d76c5ad596/monitoring-plugin/0.log"
Jul 07 17:48:48.927878 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:48.927858    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lx7tn_4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764/node-exporter/0.log"
Jul 07 17:48:48.952441 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:48.952422    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lx7tn_4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764/kube-rbac-proxy/0.log"
Jul 07 17:48:48.978187 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:48.978169    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lx7tn_4c4e36aa-1cb3-4b5b-bf45-3f05d4cc4764/init-textfile/0.log"
Jul 07 17:48:49.472654 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:49.472629    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-x68w5_35ad1132-a443-447a-bee9-3fe909270852/prometheus-operator/0.log"
Jul 07 17:48:49.499406 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:49.499384    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-x68w5_35ad1132-a443-447a-bee9-3fe909270852/kube-rbac-proxy/0.log"
Jul 07 17:48:51.036544 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:51.036516    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-47h5d/perf-node-gather-daemonset-fbmrf"]
Jul 07 17:48:51.036927 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:51.036826    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cce40d4-d28c-49d0-a627-4cd730aa2e9f" containerName="main"
Jul 07 17:48:51.036927 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:51.036838    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cce40d4-d28c-49d0-a627-4cd730aa2e9f" containerName="main"
Jul 07 17:48:51.036927 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:51.036848    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="791659eb-8ba7-43ac-9a08-ad5ff86f74ed" containerName="main"
Jul 07 17:48:51.036927 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:51.036854    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="791659eb-8ba7-43ac-9a08-ad5ff86f74ed" containerName="main"
Jul 07 17:48:51.037068 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:51.036949    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="791659eb-8ba7-43ac-9a08-ad5ff86f74ed" containerName="main"
Jul 07 17:48:51.037068 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:51.036958    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="0cce40d4-d28c-49d0-a627-4cd730aa2e9f" containerName="main"
Jul 07 17:48:51.040016 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:51.040001    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-47h5d/perf-node-gather-daemonset-fbmrf"
Jul 07 17:48:51.042413 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:51.042395    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-47h5d\"/\"default-dockercfg-mt88j\""
Jul 07 17:48:51.042856 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:51.042840    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-47h5d\"/\"kube-root-ca.crt\""
Jul 07 17:48:51.042972 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:51.042840    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-47h5d\"/\"openshift-service-ca.crt\""
Jul 07 17:48:51.049706 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:51.049687    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-47h5d/perf-node-gather-daemonset-fbmrf"]
Jul 07 17:48:51.158281 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:51.158258    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bbb7f5b6-f455-4311-bc7c-55853857bd59-proc\") pod \"perf-node-gather-daemonset-fbmrf\" (UID: \"bbb7f5b6-f455-4311-bc7c-55853857bd59\") " pod="openshift-must-gather-47h5d/perf-node-gather-daemonset-fbmrf"
Jul 07 17:48:51.158384 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:51.158294    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bbb7f5b6-f455-4311-bc7c-55853857bd59-lib-modules\") pod \"perf-node-gather-daemonset-fbmrf\" (UID: \"bbb7f5b6-f455-4311-bc7c-55853857bd59\") " pod="openshift-must-gather-47h5d/perf-node-gather-daemonset-fbmrf"
Jul 07 17:48:51.158384 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:51.158324    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bbb7f5b6-f455-4311-bc7c-55853857bd59-podres\") pod \"perf-node-gather-daemonset-fbmrf\" (UID: \"bbb7f5b6-f455-4311-bc7c-55853857bd59\") " pod="openshift-must-gather-47h5d/perf-node-gather-daemonset-fbmrf"
Jul 07 17:48:51.158499 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:51.158399    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bbb7f5b6-f455-4311-bc7c-55853857bd59-sys\") pod \"perf-node-gather-daemonset-fbmrf\" (UID: \"bbb7f5b6-f455-4311-bc7c-55853857bd59\") " pod="openshift-must-gather-47h5d/perf-node-gather-daemonset-fbmrf"
Jul 07 17:48:51.158558 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:51.158493    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvwrr\" (UniqueName: \"kubernetes.io/projected/bbb7f5b6-f455-4311-bc7c-55853857bd59-kube-api-access-hvwrr\") pod \"perf-node-gather-daemonset-fbmrf\" (UID: \"bbb7f5b6-f455-4311-bc7c-55853857bd59\") " pod="openshift-must-gather-47h5d/perf-node-gather-daemonset-fbmrf"
Jul 07 17:48:51.259797 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:51.259773    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bbb7f5b6-f455-4311-bc7c-55853857bd59-proc\") pod \"perf-node-gather-daemonset-fbmrf\" (UID: \"bbb7f5b6-f455-4311-bc7c-55853857bd59\") " pod="openshift-must-gather-47h5d/perf-node-gather-daemonset-fbmrf"
Jul 07 17:48:51.259869 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:51.259804    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bbb7f5b6-f455-4311-bc7c-55853857bd59-lib-modules\") pod \"perf-node-gather-daemonset-fbmrf\" (UID: \"bbb7f5b6-f455-4311-bc7c-55853857bd59\") " pod="openshift-must-gather-47h5d/perf-node-gather-daemonset-fbmrf"
Jul 07 17:48:51.259869 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:51.259822    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bbb7f5b6-f455-4311-bc7c-55853857bd59-podres\") pod \"perf-node-gather-daemonset-fbmrf\" (UID: \"bbb7f5b6-f455-4311-bc7c-55853857bd59\") " pod="openshift-must-gather-47h5d/perf-node-gather-daemonset-fbmrf"
Jul 07 17:48:51.259869 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:51.259847    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bbb7f5b6-f455-4311-bc7c-55853857bd59-sys\") pod \"perf-node-gather-daemonset-fbmrf\" (UID: \"bbb7f5b6-f455-4311-bc7c-55853857bd59\") " pod="openshift-must-gather-47h5d/perf-node-gather-daemonset-fbmrf"
Jul 07 17:48:51.260007 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:51.259906    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bbb7f5b6-f455-4311-bc7c-55853857bd59-proc\") pod \"perf-node-gather-daemonset-fbmrf\" (UID: \"bbb7f5b6-f455-4311-bc7c-55853857bd59\") " pod="openshift-must-gather-47h5d/perf-node-gather-daemonset-fbmrf"
Jul 07 17:48:51.260007 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:51.259944    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvwrr\" (UniqueName: \"kubernetes.io/projected/bbb7f5b6-f455-4311-bc7c-55853857bd59-kube-api-access-hvwrr\") pod \"perf-node-gather-daemonset-fbmrf\" (UID: \"bbb7f5b6-f455-4311-bc7c-55853857bd59\") " pod="openshift-must-gather-47h5d/perf-node-gather-daemonset-fbmrf"
Jul 07 17:48:51.260007 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:51.259962    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bbb7f5b6-f455-4311-bc7c-55853857bd59-sys\") pod \"perf-node-gather-daemonset-fbmrf\" (UID: \"bbb7f5b6-f455-4311-bc7c-55853857bd59\") " pod="openshift-must-gather-47h5d/perf-node-gather-daemonset-fbmrf"
Jul 07 17:48:51.260007 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:51.259969    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bbb7f5b6-f455-4311-bc7c-55853857bd59-podres\") pod \"perf-node-gather-daemonset-fbmrf\" (UID: \"bbb7f5b6-f455-4311-bc7c-55853857bd59\") " pod="openshift-must-gather-47h5d/perf-node-gather-daemonset-fbmrf"
Jul 07 17:48:51.260007 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:51.259974    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bbb7f5b6-f455-4311-bc7c-55853857bd59-lib-modules\") pod \"perf-node-gather-daemonset-fbmrf\" (UID: \"bbb7f5b6-f455-4311-bc7c-55853857bd59\") " pod="openshift-must-gather-47h5d/perf-node-gather-daemonset-fbmrf"
Jul 07 17:48:51.269071 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:51.269055    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvwrr\" (UniqueName: \"kubernetes.io/projected/bbb7f5b6-f455-4311-bc7c-55853857bd59-kube-api-access-hvwrr\") pod \"perf-node-gather-daemonset-fbmrf\" (UID: \"bbb7f5b6-f455-4311-bc7c-55853857bd59\") " pod="openshift-must-gather-47h5d/perf-node-gather-daemonset-fbmrf"
Jul 07 17:48:51.350262 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:51.350214    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-47h5d/perf-node-gather-daemonset-fbmrf"
Jul 07 17:48:51.485797 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:51.485772    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-47h5d/perf-node-gather-daemonset-fbmrf"]
Jul 07 17:48:51.487843 ip-10-0-142-103 kubenswrapper[2579]: W0707 17:48:51.487821    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podbbb7f5b6_f455_4311_bc7c_55853857bd59.slice/crio-f8f67695a36ba6ed1cbe8188b5c00615c4e777b504925d4a151b2727e8db622c WatchSource:0}: Error finding container f8f67695a36ba6ed1cbe8188b5c00615c4e777b504925d4a151b2727e8db622c: Status 404 returned error can't find the container with id f8f67695a36ba6ed1cbe8188b5c00615c4e777b504925d4a151b2727e8db622c
Jul 07 17:48:52.302787 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:52.302755    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-47h5d/perf-node-gather-daemonset-fbmrf" event={"ID":"bbb7f5b6-f455-4311-bc7c-55853857bd59","Type":"ContainerStarted","Data":"f6bea181dc137ad03568128dea4a3aba8cc4ed21b964951d6297f5b9a56d459c"}
Jul 07 17:48:52.303285 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:52.302811    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-47h5d/perf-node-gather-daemonset-fbmrf" event={"ID":"bbb7f5b6-f455-4311-bc7c-55853857bd59","Type":"ContainerStarted","Data":"f8f67695a36ba6ed1cbe8188b5c00615c4e777b504925d4a151b2727e8db622c"}
Jul 07 17:48:52.303285 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:52.302876    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-47h5d/perf-node-gather-daemonset-fbmrf"
Jul 07 17:48:53.023314 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:53.023292    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-thcnc_15180ffd-916f-4516-8bb4-967a3701c482/volume-data-source-validator/0.log"
Jul 07 17:48:54.271836 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:54.271798    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nbkp6_b04e3abe-c8da-4123-ae78-3d978812545d/dns/0.log"
Jul 07 17:48:54.296583 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:54.296566    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nbkp6_b04e3abe-c8da-4123-ae78-3d978812545d/kube-rbac-proxy/0.log"
Jul 07 17:48:54.380951 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:54.380931    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-66zv2_ecf1ac63-1ea5-4590-8d60-b199e5eff959/dns-node-resolver/0.log"
Jul 07 17:48:55.088398 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:55.088358    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-74b85b9dfc-gpbwp_7954a9ce-fc31-4880-9914-5d78f8a3bc1b/registry/0.log"
Jul 07 17:48:55.113963 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:55.113941    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-87n7p_187a8f3e-75c8-461a-a0f0-f3e3723d3a34/node-ca/0.log"
Jul 07 17:48:56.469820 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:56.469789    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-76676f7fb7-ghktc_c201225c-e095-4494-b013-694764cd0272/kube-auth-proxy/0.log"
Jul 07 17:48:57.439011 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:57.438984    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-h7sqt_72d06848-e504-4952-953f-02b4800ddf45/serve-healthcheck-canary/0.log"
Jul 07 17:48:58.011733 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:58.011709    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ph5mc_299d1c5f-8255-4e02-9cef-7a80ce5f88d7/kube-rbac-proxy/0.log"
Jul 07 17:48:58.038313 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:58.038293    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ph5mc_299d1c5f-8255-4e02-9cef-7a80ce5f88d7/exporter/0.log"
Jul 07 17:48:58.064512 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:58.064488    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ph5mc_299d1c5f-8255-4e02-9cef-7a80ce5f88d7/extractor/0.log"
Jul 07 17:48:58.320034 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:58.319987    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-47h5d/perf-node-gather-daemonset-fbmrf"
Jul 07 17:48:58.339534 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:48:58.339486    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-47h5d/perf-node-gather-daemonset-fbmrf" podStartSLOduration=7.339472269 podStartE2EDuration="7.339472269s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:48:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:48:52.329938695 +0000 UTC m=+2455.826017201" watchObservedRunningTime="2026-07-07 17:48:58.339472269 +0000 UTC m=+2461.835550776"
Jul 07 17:49:01.181413 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:49:01.181385    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-4nbnm_20330f0d-cf1b-489e-a8c7-866bfbd84e4f/server/0.log"
Jul 07 17:49:01.260225 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:49:01.260193    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-clcxv_cedceb3f-624f-4148-b0cb-529b9d1888d6/manager/0.log"
Jul 07 17:49:01.471197 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:49:01.471119    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-547959ffc4-rs7zv_6995de0d-adeb-44ea-8b4c-52d7464731bd/manager/0.log"
Jul 07 17:49:01.500192 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:49:01.500171    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-547959ffc4-rs7zv_6995de0d-adeb-44ea-8b4c-52d7464731bd/copy-manifests/0.log"
Jul 07 17:49:01.804347 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:49:01.804280    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-76676f7fb7-ghktc_c201225c-e095-4494-b013-694764cd0272/kube-auth-proxy/0.log"
Jul 07 17:49:03.022799 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:49:03.022774    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5bbf97565b-hw8dw_d3be45b7-0764-4b02-8ed1-7bae149537e5/manager/0.log"
Jul 07 17:49:03.344438 ip-10-0-142-103 kubenswrapper[2579]: I0707 17:49:03.344380    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-76676f7fb7-ghktc_c201225c-e095-4494-b013-694764cd0272/kube-auth-proxy/0.log"