Jul 07 11:09:05.981831 ip-10-0-142-39 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 07 11:09:05.981844 ip-10-0-142-39 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 07 11:09:05.981853 ip-10-0-142-39 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 07 11:09:05.982155 ip-10-0-142-39 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 07 11:09:15.993964 ip-10-0-142-39 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 07 11:09:15.993984 ip-10-0-142-39 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot b5a624eb40274086b990eb1436c44d26 --
Jul 07 11:10:50.160664 ip-10-0-142-39 systemd[1]: Starting Kubernetes Kubelet...
Jul 07 11:10:50.604775 ip-10-0-142-39 kubenswrapper[2570]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 11:10:50.604775 ip-10-0-142-39 kubenswrapper[2570]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 07 11:10:50.604775 ip-10-0-142-39 kubenswrapper[2570]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 11:10:50.604775 ip-10-0-142-39 kubenswrapper[2570]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 07 11:10:50.604775 ip-10-0-142-39 kubenswrapper[2570]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 11:10:50.605936 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.605844    2570 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 07 11:10:50.612837 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612821    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 11:10:50.612837 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612838    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 11:10:50.612909 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612841    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 11:10:50.612909 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612844    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 11:10:50.612909 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612847    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 11:10:50.612909 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612850    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 11:10:50.612909 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612854    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 11:10:50.612909 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612856    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 11:10:50.612909 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612859    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 11:10:50.612909 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612861    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 11:10:50.612909 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612864    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 11:10:50.612909 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612866    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 11:10:50.612909 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612869    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 11:10:50.612909 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612872    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 11:10:50.612909 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612874    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 11:10:50.612909 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612877    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 11:10:50.612909 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612881    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 11:10:50.612909 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612885    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 11:10:50.612909 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612896    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 11:10:50.612909 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612899    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 11:10:50.612909 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612902    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 11:10:50.613390 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612905    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 11:10:50.613390 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612908    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 11:10:50.613390 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612911    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 11:10:50.613390 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612914    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 11:10:50.613390 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612917    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 11:10:50.613390 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612920    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 11:10:50.613390 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612924    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 11:10:50.613390 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612927    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 11:10:50.613390 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612931    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 11:10:50.613390 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612934    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 11:10:50.613390 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612938    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 11:10:50.613390 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612941    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 11:10:50.613390 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612943    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 11:10:50.613390 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612946    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 11:10:50.613390 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612949    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 11:10:50.613390 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612952    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 11:10:50.613390 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612954    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 11:10:50.613390 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612957    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 11:10:50.613390 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612959    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 11:10:50.614177 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612961    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 11:10:50.614177 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612964    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 11:10:50.614177 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612966    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 11:10:50.614177 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612968    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 11:10:50.614177 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612971    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 11:10:50.614177 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612973    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 11:10:50.614177 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612976    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 11:10:50.614177 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612978    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 11:10:50.614177 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612981    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 11:10:50.614177 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612984    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 11:10:50.614177 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612986    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 11:10:50.614177 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612989    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 11:10:50.614177 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612991    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 11:10:50.614177 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612994    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 11:10:50.614177 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612997    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 11:10:50.614177 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.612999    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 11:10:50.614177 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.613002    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 11:10:50.614177 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.613004    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 11:10:50.614177 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.613006    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 11:10:50.614177 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.613010    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 11:10:50.614975 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.613012    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 11:10:50.614975 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.613015    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 11:10:50.614975 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.613017    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 11:10:50.614975 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.613022    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 11:10:50.614975 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.613025    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 11:10:50.614975 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.613028    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 11:10:50.614975 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.613031    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 11:10:50.614975 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.613034    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 11:10:50.614975 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.613037    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 11:10:50.614975 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.613039    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 11:10:50.614975 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.613042    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 11:10:50.614975 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.613044    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 11:10:50.614975 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.613047    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 11:10:50.614975 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.613049    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 11:10:50.614975 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.613052    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 11:10:50.614975 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.613054    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 11:10:50.614975 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.613057    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 11:10:50.614975 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.613059    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 11:10:50.614975 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.613062    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 11:10:50.615504 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.613064    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 11:10:50.615504 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.613067    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 11:10:50.615504 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.613070    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 11:10:50.615504 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.613072    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 11:10:50.615504 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.613075    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 11:10:50.615504 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.613077    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 11:10:50.615504 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.613081    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 11:10:50.615504 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.613084    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 11:10:50.615504 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.613087    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 11:10:50.615504 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.613089    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 11:10:50.615504 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.613092    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 11:10:50.615504 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.613094    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 11:10:50.615504 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.613097    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 11:10:50.615504 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.613099    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 11:10:50.615504 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.613102    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 11:10:50.615504 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.613104    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 11:10:50.615504 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.613107    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 11:10:50.615504 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.613110    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 11:10:50.615504 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.613112    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 11:10:50.615504 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.613115    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 11:10:50.615504 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615153    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 11:10:50.615999 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615165    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 11:10:50.615999 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615170    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 11:10:50.615999 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615173    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 11:10:50.615999 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615175    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 11:10:50.615999 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615178    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 11:10:50.615999 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615181    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 11:10:50.615999 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615184    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 11:10:50.615999 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615186    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 11:10:50.615999 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615191    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 11:10:50.615999 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615195    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 11:10:50.615999 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615198    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 11:10:50.615999 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615201    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 11:10:50.615999 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615204    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 11:10:50.615999 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615206    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 11:10:50.615999 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615209    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 11:10:50.615999 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615211    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 11:10:50.615999 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615214    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 11:10:50.615999 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615217    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 11:10:50.615999 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615219    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 11:10:50.615999 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615222    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 11:10:50.616561 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615225    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 11:10:50.616561 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615228    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 11:10:50.616561 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615231    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 11:10:50.616561 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615233    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 11:10:50.616561 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615236    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 11:10:50.616561 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615238    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 11:10:50.616561 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615241    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 11:10:50.616561 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615243    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 11:10:50.616561 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615246    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 11:10:50.616561 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615248    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 11:10:50.616561 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615250    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 11:10:50.616561 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615253    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 11:10:50.616561 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615255    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 11:10:50.616561 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615258    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 11:10:50.616561 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615260    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 11:10:50.616561 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615263    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 11:10:50.616561 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615265    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 11:10:50.616561 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615268    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 11:10:50.616561 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615270    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 11:10:50.616561 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615272    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 11:10:50.617054 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615275    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 11:10:50.617054 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615277    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 11:10:50.617054 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615281    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 11:10:50.617054 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615284    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 11:10:50.617054 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615286    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 11:10:50.617054 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615289    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 11:10:50.617054 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615291    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 11:10:50.617054 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615294    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 11:10:50.617054 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615298    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 11:10:50.617054 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615302    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 11:10:50.617054 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615304    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 11:10:50.617054 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615307    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 11:10:50.617054 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615310    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 11:10:50.617054 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615313    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 11:10:50.617054 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615315    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 11:10:50.617054 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615318    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 11:10:50.617054 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615321    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 11:10:50.617054 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615342    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 11:10:50.617054 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615345    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 11:10:50.617554 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615348    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 11:10:50.617554 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615350    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 11:10:50.617554 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615353    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 11:10:50.617554 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615355    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 11:10:50.617554 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615358    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 11:10:50.617554 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615360    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 11:10:50.617554 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615363    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 11:10:50.617554 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615365    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 11:10:50.617554 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615368    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 11:10:50.617554 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615370    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 11:10:50.617554 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615373    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 11:10:50.617554 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615375    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 11:10:50.617554 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615378    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 11:10:50.617554 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615380    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 11:10:50.617554 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615383    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 11:10:50.617554 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615386    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 11:10:50.617554 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615388    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 11:10:50.617554 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615391    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 11:10:50.617554 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615393    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 11:10:50.617554 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615396    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 11:10:50.618041 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615398    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 11:10:50.618041 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615401    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 11:10:50.618041 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615404    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 11:10:50.618041 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615407    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 11:10:50.618041 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615411    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 11:10:50.618041 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615416    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 11:10:50.618041 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615419    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 11:10:50.618041 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615422    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 11:10:50.618041 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615425    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 11:10:50.618041 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615427    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 11:10:50.618041 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615430    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 11:10:50.618041 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615432    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 11:10:50.618041 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615435    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 11:10:50.618041 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615437    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 11:10:50.618041 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615440    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 11:10:50.618041 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615442    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 11:10:50.618041 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615445    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 11:10:50.618041 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615447    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 11:10:50.618041 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615453    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 11:10:50.618537 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615844    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 11:10:50.618537 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615849    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 11:10:50.618537 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615852    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 11:10:50.618537 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615854    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 11:10:50.618537 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615857    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 11:10:50.618537 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615859    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 11:10:50.618537 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615863    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 11:10:50.618537 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615865    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 11:10:50.618537 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615870    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 11:10:50.618537 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615874    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 11:10:50.618537 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615877    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 11:10:50.618537 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615880    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 11:10:50.618537 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615883    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 11:10:50.618537 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615885    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 11:10:50.618537 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615888    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 11:10:50.618537 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615895    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 11:10:50.618537 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615898    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 11:10:50.618537 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615901    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 11:10:50.618537 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615904    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 11:10:50.619015 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615906    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 11:10:50.619015 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615908    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 11:10:50.619015 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615911    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 11:10:50.619015 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615913    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 11:10:50.619015 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615916    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 11:10:50.619015 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615918    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 11:10:50.619015 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615920    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 11:10:50.619015 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615923    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 11:10:50.619015 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615925    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 11:10:50.619015 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615928    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 11:10:50.619015 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615930    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 11:10:50.619015 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615932    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 11:10:50.619015 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615937    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 11:10:50.619015 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615940    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 11:10:50.619015 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615943    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 11:10:50.619015 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615946    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 11:10:50.619015 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615949    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 11:10:50.619015 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615952    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 11:10:50.619015 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615954    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 11:10:50.619586 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615957    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 11:10:50.619586 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615960    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 11:10:50.619586 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615962    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 11:10:50.619586 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615964    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 11:10:50.619586 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615967    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 11:10:50.619586 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615970    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 11:10:50.619586 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615973    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 11:10:50.619586 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615975    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 11:10:50.619586 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615978    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 11:10:50.619586 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615980    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 11:10:50.619586 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615983    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 11:10:50.619586 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615987    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 11:10:50.619586 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615989    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 11:10:50.619586 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615991    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 11:10:50.619586 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615994    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 11:10:50.619586 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615996    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 11:10:50.619586 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.615999    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 11:10:50.619586 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616001    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 11:10:50.619586 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616013    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 11:10:50.619586 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616016    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 11:10:50.620111 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616018    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 11:10:50.620111 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616021    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 11:10:50.620111 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616025    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 11:10:50.620111 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616028    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 11:10:50.620111 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616031    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 11:10:50.620111 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616034    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 11:10:50.620111 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616037    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 11:10:50.620111 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616040    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 11:10:50.620111 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616043    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 11:10:50.620111 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616045    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 11:10:50.620111 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616047    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 11:10:50.620111 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616050    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 11:10:50.620111 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616052    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 11:10:50.620111 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616055    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 11:10:50.620111 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616057    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 11:10:50.620111 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616060    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 11:10:50.620111 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616062    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 11:10:50.620111 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616066    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 11:10:50.620111 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616068    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 11:10:50.620111 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616071    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 11:10:50.620612 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616073    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 11:10:50.620612 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616076    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 11:10:50.620612 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616078    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 11:10:50.620612 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616080    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 11:10:50.620612 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616084    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 11:10:50.620612 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616086    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 11:10:50.620612 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616089    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 11:10:50.620612 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616092    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 11:10:50.620612 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616094    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 11:10:50.620612 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616097    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 11:10:50.620612 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616099    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 11:10:50.620612 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616102    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 11:10:50.620612 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616104    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 11:10:50.620612 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616107    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 11:10:50.620612 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616109    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 11:10:50.620612 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616112    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 11:10:50.620612 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616114    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 11:10:50.620612 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616116    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 11:10:50.620612 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616119    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 11:10:50.621065 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616122    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 11:10:50.621065 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616124    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 11:10:50.621065 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616203    2570 flags.go:64] FLAG: --address="0.0.0.0"
Jul 07 11:10:50.621065 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616211    2570 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 07 11:10:50.621065 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616216    2570 flags.go:64] FLAG: --anonymous-auth="true"
Jul 07 11:10:50.621065 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616221    2570 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 07 11:10:50.621065 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616225    2570 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 07 11:10:50.621065 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616228    2570 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 07 11:10:50.621065 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616233    2570 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 07 11:10:50.621065 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616237    2570 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 07 11:10:50.621065 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616240    2570 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 07 11:10:50.621065 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616245    2570 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 07 11:10:50.621065 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616248    2570 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 07 11:10:50.621065 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616251    2570 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 07 11:10:50.621065 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616254    2570 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 07 11:10:50.621065 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616257    2570 flags.go:64] FLAG: --cgroup-root=""
Jul 07 11:10:50.621065 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616260    2570 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 07 11:10:50.621065 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616263    2570 flags.go:64] FLAG: --client-ca-file=""
Jul 07 11:10:50.621065 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616266    2570 flags.go:64] FLAG: --cloud-provider="external"
Jul 07 11:10:50.621065 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616269    2570 flags.go:64] FLAG: --cluster-dns="[]"
Jul 07 11:10:50.621065 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616274    2570 flags.go:64] FLAG: --cluster-domain=""
Jul 07 11:10:50.621065 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616277    2570 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 07 11:10:50.621065 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616280    2570 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 07 11:10:50.621065 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616283    2570 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 07 11:10:50.621713 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616286    2570 flags.go:64] FLAG: --container-log-max-files="5"
Jul 07 11:10:50.621713 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616290    2570 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 07 11:10:50.621713 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616293    2570 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 07 11:10:50.621713 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616296    2570 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 07 11:10:50.621713 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616299    2570 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 07 11:10:50.621713 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616302    2570 flags.go:64] FLAG: --contention-profiling="false"
Jul 07 11:10:50.621713 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616305    2570 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 07 11:10:50.621713 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616307    2570 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 07 11:10:50.621713 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616310    2570 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 07 11:10:50.621713 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616313    2570 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 07 11:10:50.621713 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616317    2570 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 07 11:10:50.621713 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616320    2570 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 07 11:10:50.621713 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616335    2570 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 07 11:10:50.621713 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616338    2570 flags.go:64] FLAG: --enable-load-reader="false"
Jul 07 11:10:50.621713 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616341    2570 flags.go:64] FLAG: --enable-server="true"
Jul 07 11:10:50.621713 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616344    2570 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 07 11:10:50.621713 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616348    2570 flags.go:64] FLAG: --event-burst="100"
Jul 07 11:10:50.621713 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616351    2570 flags.go:64] FLAG: --event-qps="50"
Jul 07 11:10:50.621713 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616354    2570 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 07 11:10:50.621713 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616357    2570 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 07 11:10:50.621713 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616359    2570 flags.go:64] FLAG: --eviction-hard=""
Jul 07 11:10:50.621713 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616363    2570 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 07 11:10:50.621713 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616366    2570 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 07 11:10:50.621713 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616368    2570 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 07 11:10:50.621713 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616371    2570 flags.go:64] FLAG: --eviction-soft=""
Jul 07 11:10:50.622310 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616374    2570 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 07 11:10:50.622310 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616377    2570 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 07 11:10:50.622310 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616382    2570 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 07 11:10:50.622310 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616385    2570 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 07 11:10:50.622310 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616388    2570 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 07 11:10:50.622310 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616390    2570 flags.go:64] FLAG: --fail-swap-on="true"
Jul 07 11:10:50.622310 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616393    2570 flags.go:64] FLAG: --feature-gates=""
Jul 07 11:10:50.622310 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616396    2570 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 07 11:10:50.622310 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616399    2570 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 07 11:10:50.622310 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616402    2570 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 07 11:10:50.622310 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616405    2570 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 07 11:10:50.622310 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616408    2570 flags.go:64] FLAG: --healthz-port="10248"
Jul 07 11:10:50.622310 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616413    2570 flags.go:64] FLAG: --help="false"
Jul 07 11:10:50.622310 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616416    2570 flags.go:64] FLAG: --hostname-override="ip-10-0-142-39.ec2.internal"
Jul 07 11:10:50.622310 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616419    2570 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 07 11:10:50.622310 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616422    2570 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 07 11:10:50.622310 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616424    2570 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 07 11:10:50.622310 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616428    2570 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 07 11:10:50.622310 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616431    2570 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 07 11:10:50.622310 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616433    2570 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 07 11:10:50.622310 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616436    2570 flags.go:64] FLAG: --image-service-endpoint=""
Jul 07 11:10:50.622310 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616439    2570 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 07 11:10:50.622310 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616442    2570 flags.go:64] FLAG: --kube-api-burst="100"
Jul 07 11:10:50.622899 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616444    2570 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 07 11:10:50.622899 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616447    2570 flags.go:64] FLAG: --kube-api-qps="50"
Jul 07 11:10:50.622899 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616450    2570 flags.go:64] FLAG: --kube-reserved=""
Jul 07 11:10:50.622899 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616452    2570 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 07 11:10:50.622899 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616455    2570 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 07 11:10:50.622899 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616458    2570 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 07 11:10:50.622899 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616460    2570 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 07 11:10:50.622899 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616463    2570 flags.go:64] FLAG: --lock-file=""
Jul 07 11:10:50.622899 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616466    2570 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 07 11:10:50.622899 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616469    2570 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 07 11:10:50.622899 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616472    2570 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 07 11:10:50.622899 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616477    2570 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 07 11:10:50.622899 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616482    2570 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 07 11:10:50.622899 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616485    2570 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 07 11:10:50.622899 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616487    2570 flags.go:64] FLAG: --logging-format="text"
Jul 07 11:10:50.622899 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616490    2570 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 07 11:10:50.622899 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616493    2570 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 07 11:10:50.622899 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616496    2570 flags.go:64] FLAG: --manifest-url=""
Jul 07 11:10:50.622899 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616499    2570 flags.go:64] FLAG: --manifest-url-header=""
Jul 07 11:10:50.622899 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616503    2570 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 07 11:10:50.622899 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616506    2570 flags.go:64] FLAG: --max-open-files="1000000"
Jul 07 11:10:50.622899 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616511    2570 flags.go:64] FLAG: --max-pods="110"
Jul 07 11:10:50.622899 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616514    2570 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 07 11:10:50.622899 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616516    2570 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 07 11:10:50.622899 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616519    2570 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 07 11:10:50.623537 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616522    2570 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 07 11:10:50.623537 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616525    2570 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 07 11:10:50.623537 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616527    2570 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 07 11:10:50.623537 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616530    2570 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 07 11:10:50.623537 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616537    2570 flags.go:64] FLAG: --node-status-max-images="50"
Jul 07 11:10:50.623537 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616540    2570 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 07 11:10:50.623537 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616543    2570 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 07 11:10:50.623537 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616546    2570 flags.go:64] FLAG: --pod-cidr=""
Jul 07 11:10:50.623537 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616549    2570 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 07 11:10:50.623537 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616555    2570 flags.go:64] FLAG: --pod-manifest-path=""
Jul 07 11:10:50.623537 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616557    2570 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 07 11:10:50.623537 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616560    2570 flags.go:64] FLAG: --pods-per-core="0"
Jul 07 11:10:50.623537 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616563    2570 flags.go:64] FLAG: --port="10250"
Jul 07 11:10:50.623537 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616566    2570 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 07 11:10:50.623537 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616569    2570 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-00c39338ed24aa6e1"
Jul 07 11:10:50.623537 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616572    2570 flags.go:64] FLAG: --qos-reserved=""
Jul 07 11:10:50.623537 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616576    2570 flags.go:64] FLAG: --read-only-port="10255"
Jul 07 11:10:50.623537 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616578    2570 flags.go:64] FLAG: --register-node="true"
Jul 07 11:10:50.623537 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616581    2570 flags.go:64] FLAG: --register-with-taints=""
Jul 07 11:10:50.623537 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616584    2570 flags.go:64] FLAG: --registry-burst="10"
Jul 07 11:10:50.623537 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616589    2570 flags.go:64] FLAG: --registry-qps="5"
Jul 07 11:10:50.623537 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616591    2570 flags.go:64] FLAG: --reserved-cpus=""
Jul 07 11:10:50.623537 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616594    2570 flags.go:64] FLAG: --reserved-memory=""
Jul 07 11:10:50.623537 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616602    2570 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 07 11:10:50.623537 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616605    2570 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 07 11:10:50.624172 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616608    2570 flags.go:64] FLAG: --rotate-certificates="false"
Jul 07 11:10:50.624172 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616611    2570 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 07 11:10:50.624172 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616613    2570 flags.go:64] FLAG: --runonce="false"
Jul 07 11:10:50.624172 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616617    2570 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 07 11:10:50.624172 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616620    2570 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 07 11:10:50.624172 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616623    2570 flags.go:64] FLAG: --seccomp-default="false"
Jul 07 11:10:50.624172 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616626    2570 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 07 11:10:50.624172 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616629    2570 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 07 11:10:50.624172 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616631    2570 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 07 11:10:50.624172 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616634    2570 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 07 11:10:50.624172 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616637    2570 flags.go:64] FLAG: --storage-driver-password="root"
Jul 07 11:10:50.624172 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616640    2570 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 07 11:10:50.624172 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616643    2570 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 07 11:10:50.624172 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616646    2570 flags.go:64] FLAG: --storage-driver-user="root"
Jul 07 11:10:50.624172 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616648    2570 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 07 11:10:50.624172 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616651    2570 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 07 11:10:50.624172 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616654    2570 flags.go:64] FLAG: --system-cgroups=""
Jul 07 11:10:50.624172 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616657    2570 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 07 11:10:50.624172 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616662    2570 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 07 11:10:50.624172 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616664    2570 flags.go:64] FLAG: --tls-cert-file=""
Jul 07 11:10:50.624172 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616667    2570 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 07 11:10:50.624172 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616671    2570 flags.go:64] FLAG: --tls-min-version=""
Jul 07 11:10:50.624172 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616674    2570 flags.go:64] FLAG: --tls-private-key-file=""
Jul 07 11:10:50.624172 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616677    2570 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 07 11:10:50.624172 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616679    2570 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 07 11:10:50.624798 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616682    2570 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 07 11:10:50.624798 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616684    2570 flags.go:64] FLAG: --v="2"
Jul 07 11:10:50.624798 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616690    2570 flags.go:64] FLAG: --version="false"
Jul 07 11:10:50.624798 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616694    2570 flags.go:64] FLAG: --vmodule=""
Jul 07 11:10:50.624798 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616698    2570 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 07 11:10:50.624798 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.616701    2570 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 07 11:10:50.624798 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616823    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 11:10:50.624798 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616827    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 11:10:50.624798 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616831    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 11:10:50.624798 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616835    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 11:10:50.624798 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616839    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 11:10:50.624798 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616842    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 11:10:50.624798 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616844    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 11:10:50.624798 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616848    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 11:10:50.624798 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616850    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 11:10:50.624798 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616853    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 11:10:50.624798 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616855    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 11:10:50.624798 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616858    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 11:10:50.624798 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616860    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 11:10:50.624798 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616863    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 11:10:50.624798 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616865    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 11:10:50.625396 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616868    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 11:10:50.625396 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616870    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 11:10:50.625396 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616873    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 11:10:50.625396 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616875    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 11:10:50.625396 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616878    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 11:10:50.625396 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616880    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 11:10:50.625396 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616883    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 11:10:50.625396 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616885    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 11:10:50.625396 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616888    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 11:10:50.625396 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616890    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 11:10:50.625396 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616892    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 11:10:50.625396 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616895    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 11:10:50.625396 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616898    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 11:10:50.625396 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616901    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 11:10:50.625396 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616904    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 11:10:50.625396 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616907    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 11:10:50.625396 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616910    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 11:10:50.625396 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616912    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 11:10:50.625396 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616915    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 11:10:50.625396 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616918    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 11:10:50.625885 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616921    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 11:10:50.625885 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616924    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 11:10:50.625885 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616927    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 11:10:50.625885 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616929    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 11:10:50.625885 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616932    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 11:10:50.625885 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616934    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 11:10:50.625885 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616938    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 11:10:50.625885 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616942    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 11:10:50.625885 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616945    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 11:10:50.625885 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616947    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 11:10:50.625885 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616950    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 11:10:50.625885 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616953    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 11:10:50.625885 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616955    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 11:10:50.625885 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616958    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 11:10:50.625885 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616960    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 11:10:50.625885 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616963    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 11:10:50.625885 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616966    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 11:10:50.625885 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616968    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 11:10:50.625885 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616971    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 11:10:50.626411 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616973    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 11:10:50.626411 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616976    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 11:10:50.626411 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616978    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 11:10:50.626411 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616980    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 11:10:50.626411 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616983    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 11:10:50.626411 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616985    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 11:10:50.626411 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616990    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 11:10:50.626411 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616992    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 11:10:50.626411 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616995    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 11:10:50.626411 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.616998    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 11:10:50.626411 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.617000    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 11:10:50.626411 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.617003    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 11:10:50.626411 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.617006    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 11:10:50.626411 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.617008    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 11:10:50.626411 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.617012    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 11:10:50.626411 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.617015    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 11:10:50.626411 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.617017    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 11:10:50.626411 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.617020    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 11:10:50.626411 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.617022    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 11:10:50.626869 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.617026    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 11:10:50.626869 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.617029    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 11:10:50.626869 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.617032    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 11:10:50.626869 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.617035    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 11:10:50.626869 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.617038    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 11:10:50.626869 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.617041    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 11:10:50.626869 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.617043    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 11:10:50.626869 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.617045    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 11:10:50.626869 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.617048    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 11:10:50.626869 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.617050    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 11:10:50.626869 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.617053    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 11:10:50.626869 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.617055    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 11:10:50.626869 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.617058    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 11:10:50.626869 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.617060    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 11:10:50.626869 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.617063    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 11:10:50.626869 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.617066    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 11:10:50.626869 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.617069    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 11:10:50.626869 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.617071    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 11:10:50.626869 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.617074    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 11:10:50.627321 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.617077    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 11:10:50.627321 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.617080    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 11:10:50.627321 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.617082    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 11:10:50.627321 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.617085    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 11:10:50.627321 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.617088    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 11:10:50.627321 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.617090    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 11:10:50.627321 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.617093    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 11:10:50.627321 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.617775    2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 11:10:50.627321 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.625152    2570 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 07 11:10:50.627321 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.625168    2570 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 07 11:10:50.627321 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.626120    2570 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 07 11:10:50.627321 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.626127    2570 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 07 11:10:50.627321 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626190    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 11:10:50.627321 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626195    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 11:10:50.627321 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626199    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 11:10:50.627709 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626202    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 11:10:50.627709 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626206    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 11:10:50.627709 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626209    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 11:10:50.627709 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626213    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 11:10:50.627709 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626217    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 11:10:50.627709 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626220    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 11:10:50.627709 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626223    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 11:10:50.627709 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626226    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 11:10:50.627709 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626228    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 11:10:50.627709 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626231    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 11:10:50.627709 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626234    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 11:10:50.627709 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626236    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 11:10:50.627709 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626239    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 11:10:50.627709 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626241    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 11:10:50.627709 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626244    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 11:10:50.627709 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626247    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 11:10:50.627709 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626249    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 11:10:50.627709 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626252    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 11:10:50.627709 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626254    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 11:10:50.628183 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626257    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 11:10:50.628183 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626259    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 11:10:50.628183 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626263    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 11:10:50.628183 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626266    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 11:10:50.628183 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626269    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 11:10:50.628183 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626271    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 11:10:50.628183 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626274    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 11:10:50.628183 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626277    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 11:10:50.628183 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626280    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 11:10:50.628183 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626282    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 11:10:50.628183 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626285    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 11:10:50.628183 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626287    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 11:10:50.628183 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626290    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 11:10:50.628183 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626294    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 11:10:50.628183 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626298    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 11:10:50.628183 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626301    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 11:10:50.628183 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626304    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 11:10:50.628183 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626307    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 11:10:50.628183 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626309    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 11:10:50.628659 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626311    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 11:10:50.628659 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626314    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 11:10:50.628659 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626317    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 11:10:50.628659 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626319    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 11:10:50.628659 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626334    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 11:10:50.628659 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626337    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 11:10:50.628659 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626339    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 11:10:50.628659 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626342    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 11:10:50.628659 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626345    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 11:10:50.628659 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626347    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 11:10:50.628659 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626350    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 11:10:50.628659 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626352    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 11:10:50.628659 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626355    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 11:10:50.628659 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626358    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 11:10:50.628659 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626360    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 11:10:50.628659 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626363    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 11:10:50.628659 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626366    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 11:10:50.628659 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626369    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 11:10:50.628659 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626371    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 11:10:50.628659 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626374    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 11:10:50.629134 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626377    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 11:10:50.629134 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626380    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 11:10:50.629134 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626383    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 11:10:50.629134 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626385    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 11:10:50.629134 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626388    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 11:10:50.629134 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626391    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 11:10:50.629134 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626393    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 11:10:50.629134 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626396    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 11:10:50.629134 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626399    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 11:10:50.629134 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626401    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 11:10:50.629134 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626405    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 11:10:50.629134 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626408    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 11:10:50.629134 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626411    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 11:10:50.629134 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626414    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 11:10:50.629134 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626416    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 11:10:50.629134 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626419    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 11:10:50.629134 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626422    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 11:10:50.629134 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626425    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 11:10:50.629134 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626428    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 11:10:50.629605 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626430    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 11:10:50.629605 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626433    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 11:10:50.629605 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626436    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 11:10:50.629605 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626438    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 11:10:50.629605 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626441    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 11:10:50.629605 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626443    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 11:10:50.629605 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626446    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 11:10:50.629605 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626448    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 11:10:50.629605 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626451    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 11:10:50.629605 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626453    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 11:10:50.629605 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626457    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 11:10:50.629605 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626459    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 11:10:50.629605 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626462    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 11:10:50.629605 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626466    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 11:10:50.629605 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626468    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 11:10:50.629605 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626471    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 11:10:50.629605 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626473    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 11:10:50.629605 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626476    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 11:10:50.629605 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626478    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 11:10:50.630075 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.626483    2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 11:10:50.630075 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626598    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 11:10:50.630075 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626604    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 11:10:50.630075 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626607    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 11:10:50.630075 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626609    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 11:10:50.630075 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626612    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 11:10:50.630075 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626615    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 11:10:50.630075 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626618    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 11:10:50.630075 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626621    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 11:10:50.630075 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626624    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 11:10:50.630075 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626626    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 11:10:50.630075 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626629    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 11:10:50.630075 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626632    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 11:10:50.630075 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626635    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 11:10:50.630075 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626638    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 11:10:50.630525 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626640    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 11:10:50.630525 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626643    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 11:10:50.630525 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626646    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 11:10:50.630525 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626648    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 11:10:50.630525 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626651    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 11:10:50.630525 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626655    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 11:10:50.630525 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626660    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 11:10:50.630525 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626663    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 11:10:50.630525 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626666    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 11:10:50.630525 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626669    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 11:10:50.630525 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626672    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 11:10:50.630525 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626675    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 11:10:50.630525 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626678    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 11:10:50.630525 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626680    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 11:10:50.630525 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626683    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 11:10:50.630525 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626685    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 11:10:50.630525 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626687    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 11:10:50.630525 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626690    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 11:10:50.630525 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626692    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 11:10:50.630982 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626695    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 11:10:50.630982 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626697    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 11:10:50.630982 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626700    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 11:10:50.630982 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626702    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 11:10:50.630982 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626705    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 11:10:50.630982 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626707    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 11:10:50.630982 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626710    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 11:10:50.630982 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626712    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 11:10:50.630982 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626716    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 11:10:50.630982 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626718    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 11:10:50.630982 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626720    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 11:10:50.630982 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626724    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 11:10:50.630982 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626726    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 11:10:50.630982 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626728    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 11:10:50.630982 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626731    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 11:10:50.630982 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626733    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 11:10:50.630982 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626736    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 11:10:50.630982 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626738    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 11:10:50.630982 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626741    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 11:10:50.630982 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626744    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 11:10:50.631479 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626746    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 11:10:50.631479 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626749    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 11:10:50.631479 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626753    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 11:10:50.631479 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626757    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 11:10:50.631479 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626759    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 11:10:50.631479 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626762    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 11:10:50.631479 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626765    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 11:10:50.631479 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626767    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 11:10:50.631479 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626769    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 11:10:50.631479 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626772    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 11:10:50.631479 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626774    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 11:10:50.631479 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626777    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 11:10:50.631479 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626779    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 11:10:50.631479 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626781    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 11:10:50.631479 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626784    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 11:10:50.631479 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626786    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 11:10:50.631479 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626789    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 11:10:50.631479 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626791    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 11:10:50.631479 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626793    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 11:10:50.631931 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626796    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 11:10:50.631931 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626798    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 11:10:50.631931 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626801    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 11:10:50.631931 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626804    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 11:10:50.631931 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626806    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 11:10:50.631931 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626809    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 11:10:50.631931 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626811    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 11:10:50.631931 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626814    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 11:10:50.631931 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626816    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 11:10:50.631931 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626818    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 11:10:50.631931 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626821    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 11:10:50.631931 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626823    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 11:10:50.631931 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626827    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 11:10:50.631931 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626830    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 11:10:50.631931 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626833    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 11:10:50.631931 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626836    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 11:10:50.631931 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626839    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 11:10:50.631931 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626843    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 11:10:50.631931 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626845    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 11:10:50.631931 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626848    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 11:10:50.632421 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626850    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 11:10:50.632421 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626853    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 11:10:50.632421 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626855    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 11:10:50.632421 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626858    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 11:10:50.632421 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626860    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 11:10:50.632421 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626862    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 11:10:50.632421 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:50.626865    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 11:10:50.632421 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.626870    2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 11:10:50.632421 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.626990    2570 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 07 11:10:50.632421 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.630042    2570 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 07 11:10:50.632421 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.631689    2570 server.go:1019] "Starting client certificate rotation"
Jul 07 11:10:50.632421 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.631794    2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 07 11:10:50.632421 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.631829    2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 07 11:10:50.634669 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.634656    2570 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 07 11:10:50.636594 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.636579    2570 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 07 11:10:50.648084 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.648063    2570 log.go:25] "Validated CRI v1 runtime API"
Jul 07 11:10:50.650946 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.650922    2570 log.go:25] "Validated CRI v1 image API"
Jul 07 11:10:50.653337 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.653306    2570 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 07 11:10:50.656764 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.656745    2570 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 aa45d5d7-2e08-4121-bf0a-7e731f610164:/dev/nvme0n1p3 c1648745-4930-4474-9554-d2c6f958bcc0:/dev/nvme0n1p4]
Jul 07 11:10:50.656839 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.656764    2570 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 07 11:10:50.662880 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.662760    2570 manager.go:216] Machine: {Timestamp:2026-07-07 11:10:50.661284163 +0000 UTC m=+0.371180542 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3105157 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e61058287e4a1a13326162c45f476 SystemUUID:ec2e6105-8287-e4a1-a133-26162c45f476 BootID:b5a624eb-4027-4086-b990-eb1436c44d26 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:1d:ca:7b:89:31 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:1d:ca:7b:89:31 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:a2:b2:db:6b:e5:e0 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 07 11:10:50.662880 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.662872    2570 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 07 11:10:50.663044 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.662955    2570 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 07 11:10:50.665201 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.665182    2570 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 07 11:10:50.666472 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.666447    2570 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 07 11:10:50.666639 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.666479    2570 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-39.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 07 11:10:50.667390 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.667377    2570 topology_manager.go:138] "Creating topology manager with none policy"
Jul 07 11:10:50.667450 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.667395    2570 container_manager_linux.go:309] "Creating device plugin manager"
Jul 07 11:10:50.667450 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.667405    2570 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 07 11:10:50.667450 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.667427    2570 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 07 11:10:50.669243 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.669231    2570 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 07 11:10:50.673865 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.673851    2570 state_mem.go:36] "Initialized new in-memory state store"
Jul 07 11:10:50.674004 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.673992    2570 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 07 11:10:50.674058 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.674030    2570 kubelet.go:486] "Attempting to sync node with API server"
Jul 07 11:10:50.674058 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.674045    2570 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 07 11:10:50.674148 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.674063    2570 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 07 11:10:50.674148 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.674080    2570 kubelet.go:398] "Adding apiserver pod source"
Jul 07 11:10:50.674148 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.674094    2570 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 07 11:10:50.674512 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.674497    2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 07 11:10:50.674587 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.674523    2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 07 11:10:50.677224 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.677209    2570 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 07 11:10:50.679615 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.679599    2570 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 07 11:10:50.679698 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.679644    2570 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 07 11:10:50.681170 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.681159    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 07 11:10:50.681225 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.681175    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 07 11:10:50.681225 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.681182    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 07 11:10:50.681225 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.681192    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 07 11:10:50.681225 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.681206    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 07 11:10:50.681225 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.681212    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 07 11:10:50.681225 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.681222    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 07 11:10:50.681225 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.681227    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 07 11:10:50.681432 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.681234    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 07 11:10:50.681432 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.681240    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 07 11:10:50.681432 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.681254    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 07 11:10:50.681432 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.681259    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 07 11:10:50.682137 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.682127    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 07 11:10:50.682184 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.682140    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 07 11:10:50.685862 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.685847    2570 server.go:1268] "Started kubelet"
Jul 07 11:10:50.685961 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.685933    2570 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 07 11:10:50.686273 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.686055    2570 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 07 11:10:50.686927 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.686536    2570 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 07 11:10:50.686927 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:50.686701    2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-39.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 07 11:10:50.686927 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.686739    2570 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-39.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 07 11:10:50.686927 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:50.686826    2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 07 11:10:50.687008 ip-10-0-142-39 systemd[1]: Started Kubernetes Kubelet.
Jul 07 11:10:50.688770 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.688738    2570 server.go:310] "Adding debug handlers to kubelet server"
Jul 07 11:10:50.690884 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.688945    2570 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 07 11:10:50.694500 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:50.691079    2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-39.ec2.internal.18bffd73aa837f39  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-39.ec2.internal,UID:ip-10-0-142-39.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-39.ec2.internal,},FirstTimestamp:2026-07-07 11:10:50.685816633 +0000 UTC m=+0.395713010,LastTimestamp:2026-07-07 11:10:50.685816633 +0000 UTC m=+0.395713010,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-39.ec2.internal,}"
Jul 07 11:10:50.695845 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:50.695823    2570 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 07 11:10:50.701395 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.701375    2570 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 07 11:10:50.701483 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.701402    2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 07 11:10:50.701570 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:50.701551    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-39.ec2.internal\" not found"
Jul 07 11:10:50.702189 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.702165    2570 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 07 11:10:50.702189 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.702174    2570 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 07 11:10:50.702189 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.702196    2570 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 07 11:10:50.702533 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.702317    2570 reconstruct.go:97] "Volume reconstruction finished"
Jul 07 11:10:50.702533 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.702354    2570 reconciler.go:29] "Reconciler: start to sync state"
Jul 07 11:10:50.702533 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.702481    2570 factory.go:153] Registering CRI-O factory
Jul 07 11:10:50.702533 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.702493    2570 factory.go:223] Registration of the crio container factory successfully
Jul 07 11:10:50.702702 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.702553    2570 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 07 11:10:50.702702 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.702565    2570 factory.go:55] Registering systemd factory
Jul 07 11:10:50.702702 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.702574    2570 factory.go:223] Registration of the systemd container factory successfully
Jul 07 11:10:50.702702 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.702597    2570 factory.go:103] Registering Raw factory
Jul 07 11:10:50.702702 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.702609    2570 manager.go:1196] Started watching for new ooms in manager
Jul 07 11:10:50.703009 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.702991    2570 manager.go:349] Starting recovery of all containers
Jul 07 11:10:50.704498 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:50.704295    2570 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 07 11:10:50.704577 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:50.704521    2570 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-39.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 07 11:10:50.714071 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.714009    2570 manager.go:354] Recovery completed
Jul 07 11:10:50.720076 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.720060    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 11:10:50.723869 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.723833    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-39.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 11:10:50.723961 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.723905    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-39.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 11:10:50.723961 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.723927    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-39.ec2.internal" event="NodeHasSufficientPID"
Jul 07 11:10:50.725016 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.724794    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 11:10:50.725016 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.724742    2570 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 07 11:10:50.725016 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.724848    2570 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 07 11:10:50.725016 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.724865    2570 state_mem.go:36] "Initialized new in-memory state store"
Jul 07 11:10:50.726539 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.726525    2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rmcnf"
Jul 07 11:10:50.727422 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.727412    2570 policy_none.go:49] "None policy: Start"
Jul 07 11:10:50.727461 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.727426    2570 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 07 11:10:50.727461 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.727436    2570 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 07 11:10:50.729454 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.729444    2570 policy_none.go:47] "Start"
Jul 07 11:10:50.732395 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.732378    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-39.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 11:10:50.732472 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.732410    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-39.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 11:10:50.732472 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.732433    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-39.ec2.internal" event="NodeHasSufficientPID"
Jul 07 11:10:50.734071 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.734053    2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rmcnf"
Jul 07 11:10:50.764000 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.763976    2570 manager.go:337] "Starting Device Plugin manager"
Jul 07 11:10:50.782674 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:50.764035    2570 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 07 11:10:50.782674 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.764048    2570 server.go:87] "Starting device plugin registration server"
Jul 07 11:10:50.782674 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.764370    2570 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 07 11:10:50.782674 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.764383    2570 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 07 11:10:50.782674 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.764545    2570 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 07 11:10:50.782674 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.764627    2570 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 07 11:10:50.782674 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.764634    2570 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 07 11:10:50.782674 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:50.765172    2570 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 07 11:10:50.782674 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:50.765215    2570 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-39.ec2.internal\" not found"
Jul 07 11:10:50.812916 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.812837    2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 07 11:10:50.814212 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.814198    2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 07 11:10:50.814273 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.814233    2570 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 07 11:10:50.814273 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.814266    2570 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 07 11:10:50.814382 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:50.814299    2570 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 07 11:10:50.817167 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.817146    2570 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 11:10:50.865283 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.865251    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 11:10:50.866473 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.866457    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-39.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 11:10:50.866563 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.866488    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-39.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 11:10:50.866563 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.866500    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-39.ec2.internal" event="NodeHasSufficientPID"
Jul 07 11:10:50.866563 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.866527    2570 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-39.ec2.internal"
Jul 07 11:10:50.879212 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.879191    2570 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-39.ec2.internal"
Jul 07 11:10:50.879297 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:50.879215    2570 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-39.ec2.internal\": node \"ip-10-0-142-39.ec2.internal\" not found"
Jul 07 11:10:50.914694 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.914655    2570 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-39.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-39.ec2.internal"]
Jul 07 11:10:50.914875 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.914744    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 11:10:50.915703 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.915682    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-39.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 11:10:50.915807 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.915714    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-39.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 11:10:50.915807 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.915723    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-39.ec2.internal" event="NodeHasSufficientPID"
Jul 07 11:10:50.918120 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:50.918104    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-39.ec2.internal\" not found"
Jul 07 11:10:50.918177 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.918118    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 11:10:50.918280 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.918266    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-39.ec2.internal"
Jul 07 11:10:50.918365 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.918305    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 11:10:50.918880 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.918863    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-39.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 11:10:50.918880 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.918873    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-39.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 11:10:50.918993 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.918890    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-39.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 11:10:50.918993 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.918898    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-39.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 11:10:50.918993 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.918915    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-39.ec2.internal" event="NodeHasSufficientPID"
Jul 07 11:10:50.918993 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.918900    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-39.ec2.internal" event="NodeHasSufficientPID"
Jul 07 11:10:50.921275 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.921261    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-39.ec2.internal"
Jul 07 11:10:50.921367 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.921285    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 11:10:50.921929 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.921917    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-39.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 11:10:50.921999 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.921941    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-39.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 11:10:50.921999 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:50.921950    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-39.ec2.internal" event="NodeHasSufficientPID"
Jul 07 11:10:50.946954 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:50.946932    2570 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-39.ec2.internal\" not found" node="ip-10-0-142-39.ec2.internal"
Jul 07 11:10:50.951345 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:50.951310    2570 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-39.ec2.internal\" not found" node="ip-10-0-142-39.ec2.internal"
Jul 07 11:10:51.004210 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:51.004177    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/53f76728005a66da1fcc175f0398436c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-39.ec2.internal\" (UID: \"53f76728005a66da1fcc175f0398436c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-39.ec2.internal"
Jul 07 11:10:51.004210 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:51.004207    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7d767ad34fcd7349932e3bffab974a69-config\") pod \"kube-apiserver-proxy-ip-10-0-142-39.ec2.internal\" (UID: \"7d767ad34fcd7349932e3bffab974a69\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-39.ec2.internal"
Jul 07 11:10:51.004430 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:51.004226    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/53f76728005a66da1fcc175f0398436c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-39.ec2.internal\" (UID: \"53f76728005a66da1fcc175f0398436c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-39.ec2.internal"
Jul 07 11:10:51.018827 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:51.018801    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-39.ec2.internal\" not found"
Jul 07 11:10:51.104829 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:51.104739    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/53f76728005a66da1fcc175f0398436c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-39.ec2.internal\" (UID: \"53f76728005a66da1fcc175f0398436c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-39.ec2.internal"
Jul 07 11:10:51.104829 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:51.104777    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/53f76728005a66da1fcc175f0398436c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-39.ec2.internal\" (UID: \"53f76728005a66da1fcc175f0398436c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-39.ec2.internal"
Jul 07 11:10:51.104829 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:51.104795    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7d767ad34fcd7349932e3bffab974a69-config\") pod \"kube-apiserver-proxy-ip-10-0-142-39.ec2.internal\" (UID: \"7d767ad34fcd7349932e3bffab974a69\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-39.ec2.internal"
Jul 07 11:10:51.104829 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:51.104824    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/53f76728005a66da1fcc175f0398436c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-39.ec2.internal\" (UID: \"53f76728005a66da1fcc175f0398436c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-39.ec2.internal"
Jul 07 11:10:51.105060 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:51.104839    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7d767ad34fcd7349932e3bffab974a69-config\") pod \"kube-apiserver-proxy-ip-10-0-142-39.ec2.internal\" (UID: \"7d767ad34fcd7349932e3bffab974a69\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-39.ec2.internal"
Jul 07 11:10:51.105060 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:51.104845    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/53f76728005a66da1fcc175f0398436c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-39.ec2.internal\" (UID: \"53f76728005a66da1fcc175f0398436c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-39.ec2.internal"
Jul 07 11:10:51.119874 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:51.119846    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-39.ec2.internal\" not found"
Jul 07 11:10:51.220640 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:51.220596    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-39.ec2.internal\" not found"
Jul 07 11:10:51.250782 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:51.250756    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-39.ec2.internal"
Jul 07 11:10:51.254355 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:51.254303    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-39.ec2.internal"
Jul 07 11:10:51.321714 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:51.321672    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-39.ec2.internal\" not found"
Jul 07 11:10:51.422313 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:51.422249    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-39.ec2.internal\" not found"
Jul 07 11:10:51.522737 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:51.522708    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-39.ec2.internal\" not found"
Jul 07 11:10:51.623438 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:51.623383    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-39.ec2.internal\" not found"
Jul 07 11:10:51.630556 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:51.630535    2570 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 07 11:10:51.630695 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:51.630675    2570 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 07 11:10:51.724381 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:51.724277    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-39.ec2.internal\" not found"
Jul 07 11:10:51.737507 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:51.737459    2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 11:05:50 +0000 UTC" deadline="2028-04-16 21:06:48.537831148 +0000 UTC"
Jul 07 11:10:51.737507 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:51.737503    2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15585h55m56.800331515s"
Jul 07 11:10:51.824602 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:51.824572    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-39.ec2.internal\" not found"
Jul 07 11:10:51.873954 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:51.873925    2570 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 11:10:51.922675 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:51.922627    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53f76728005a66da1fcc175f0398436c.slice/crio-c9f177c5ca73098d06ec72fabcafd53fddefde66e539195cf9c5d357bcfe20ba WatchSource:0}: Error finding container c9f177c5ca73098d06ec72fabcafd53fddefde66e539195cf9c5d357bcfe20ba: Status 404 returned error can't find the container with id c9f177c5ca73098d06ec72fabcafd53fddefde66e539195cf9c5d357bcfe20ba
Jul 07 11:10:51.924764 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:51.924745    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-39.ec2.internal\" not found"
Jul 07 11:10:51.935683 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:51.935654    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 11:10:51.946790 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:51.946766    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d767ad34fcd7349932e3bffab974a69.slice/crio-e851dcb1c8565c109a36436f499ca0f8f29e5708d3b7e24afe85d10d510d5a39 WatchSource:0}: Error finding container e851dcb1c8565c109a36436f499ca0f8f29e5708d3b7e24afe85d10d510d5a39: Status 404 returned error can't find the container with id e851dcb1c8565c109a36436f499ca0f8f29e5708d3b7e24afe85d10d510d5a39
Jul 07 11:10:52.024902 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:52.024795    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-39.ec2.internal\" not found"
Jul 07 11:10:52.118662 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.118627    2570 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 11:10:52.201776 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.201738    2570 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-39.ec2.internal"
Jul 07 11:10:52.212931 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.212909    2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 07 11:10:52.214652 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.214634    2570 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-39.ec2.internal"
Jul 07 11:10:52.228016 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.227988    2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 07 11:10:52.258289 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.258255    2570 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 11:10:52.675315 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.675279    2570 apiserver.go:52] "Watching apiserver"
Jul 07 11:10:52.685450 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.685412    2570 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 07 11:10:52.686859 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.686813    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-px74n","kube-system/kube-apiserver-proxy-ip-10-0-142-39.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97g4h","openshift-cluster-node-tuning-operator/tuned-gb274","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-39.ec2.internal","openshift-multus/multus-additional-cni-plugins-gm84h","openshift-multus/network-metrics-daemon-m7hkl","openshift-network-operator/iptables-alerter-x6xq9","kube-system/konnectivity-agent-dxztq","openshift-dns/node-resolver-rqrgc","openshift-image-registry/node-ca-x8c7r","openshift-multus/multus-bg78n","openshift-network-diagnostics/network-check-target-n5mhp","openshift-ovn-kubernetes/ovnkube-node-dbsg9"]
Jul 07 11:10:52.690523 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.690062    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m7hkl"
Jul 07 11:10:52.690523 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:52.690194    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m7hkl" podUID="38a7b9ab-6691-41f8-a3b7-e47cec808b30"
Jul 07 11:10:52.692228 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.692202    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-x8c7r"
Jul 07 11:10:52.694749 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.694724    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97g4h"
Jul 07 11:10:52.696181 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.696157    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-fs9bb\""
Jul 07 11:10:52.696285 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.696229    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 07 11:10:52.696535 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.696516    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 07 11:10:52.696619 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.696558    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 07 11:10:52.700024 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.699994    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gb274"
Jul 07 11:10:52.701843 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.701824    2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 07 11:10:52.704285 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.703110    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.704285 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.703505    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 07 11:10:52.704285 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.703633    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-wq59m\""
Jul 07 11:10:52.704285 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.703808    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 07 11:10:52.704285 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.704084    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 07 11:10:52.704285 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.704128    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 07 11:10:52.704285 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.704139    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-h59f8\""
Jul 07 11:10:52.704908 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.704891    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 07 11:10:52.705485 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.705463    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gm84h"
Jul 07 11:10:52.706787 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.706688    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 07 11:10:52.710273 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.708055    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 07 11:10:52.710273 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.708582    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dxztq"
Jul 07 11:10:52.710273 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.709239    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 07 11:10:52.710273 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.709776    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 07 11:10:52.710273 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.709950    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qqpsk\""
Jul 07 11:10:52.710273 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.710217    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 07 11:10:52.710273 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.710227    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 07 11:10:52.710788 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.710292    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 07 11:10:52.710788 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.710649    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 07 11:10:52.710788 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.710745    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 07 11:10:52.710788 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.710773    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 07 11:10:52.710978 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.710835    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 07 11:10:52.710978 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.710911    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-n99rt\""
Jul 07 11:10:52.711084 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.711039    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5mhp"
Jul 07 11:10:52.711139 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:52.711105    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n5mhp" podUID="f971d15b-e001-45d9-ba6f-e0b60efc55cc"
Jul 07 11:10:52.713950 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.713899    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-x6xq9"
Jul 07 11:10:52.715708 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.715681    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9386a349-2fad-4a2c-88e0-45c997cf3e6b-socket-dir\") pod \"aws-ebs-csi-driver-node-97g4h\" (UID: \"9386a349-2fad-4a2c-88e0-45c997cf3e6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97g4h"
Jul 07 11:10:52.715810 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.715724    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/58b3f62f-e190-4a7c-9a48-c7329e02c146-var-lib-kubelet\") pod \"tuned-gb274\" (UID: \"58b3f62f-e190-4a7c-9a48-c7329e02c146\") " pod="openshift-cluster-node-tuning-operator/tuned-gb274"
Jul 07 11:10:52.715810 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.715752    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-host-run-netns\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.715810 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.715775    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-run-systemd\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.715810 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.715800    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9386a349-2fad-4a2c-88e0-45c997cf3e6b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-97g4h\" (UID: \"9386a349-2fad-4a2c-88e0-45c997cf3e6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97g4h"
Jul 07 11:10:52.715987 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.715830    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9386a349-2fad-4a2c-88e0-45c997cf3e6b-device-dir\") pod \"aws-ebs-csi-driver-node-97g4h\" (UID: \"9386a349-2fad-4a2c-88e0-45c997cf3e6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97g4h"
Jul 07 11:10:52.715987 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.715860    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-host-slash\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.715987 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.715883    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-run-openvswitch\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.715987 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.715908    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-log-socket\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.715987 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.715934    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.715987 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.715976    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-node-log\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.716170 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.716007    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-env-overrides\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.716170 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.716031    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-ovn-node-metrics-cert\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.716170 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.716057    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9386a349-2fad-4a2c-88e0-45c997cf3e6b-registration-dir\") pod \"aws-ebs-csi-driver-node-97g4h\" (UID: \"9386a349-2fad-4a2c-88e0-45c997cf3e6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97g4h"
Jul 07 11:10:52.716170 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.716083    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9386a349-2fad-4a2c-88e0-45c997cf3e6b-etc-selinux\") pod \"aws-ebs-csi-driver-node-97g4h\" (UID: \"9386a349-2fad-4a2c-88e0-45c997cf3e6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97g4h"
Jul 07 11:10:52.716170 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.716107    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/58b3f62f-e190-4a7c-9a48-c7329e02c146-etc-systemd\") pod \"tuned-gb274\" (UID: \"58b3f62f-e190-4a7c-9a48-c7329e02c146\") " pod="openshift-cluster-node-tuning-operator/tuned-gb274"
Jul 07 11:10:52.716170 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.716158    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/58b3f62f-e190-4a7c-9a48-c7329e02c146-run\") pod \"tuned-gb274\" (UID: \"58b3f62f-e190-4a7c-9a48-c7329e02c146\") " pod="openshift-cluster-node-tuning-operator/tuned-gb274"
Jul 07 11:10:52.716348 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.716185    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/58b3f62f-e190-4a7c-9a48-c7329e02c146-tmp\") pod \"tuned-gb274\" (UID: \"58b3f62f-e190-4a7c-9a48-c7329e02c146\") " pod="openshift-cluster-node-tuning-operator/tuned-gb274"
Jul 07 11:10:52.716348 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.716234    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bk9f2\" (UniqueName: \"kubernetes.io/projected/58b3f62f-e190-4a7c-9a48-c7329e02c146-kube-api-access-bk9f2\") pod \"tuned-gb274\" (UID: \"58b3f62f-e190-4a7c-9a48-c7329e02c146\") " pod="openshift-cluster-node-tuning-operator/tuned-gb274"
Jul 07 11:10:52.716348 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.716270    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-etc-openvswitch\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.716348 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.716300    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9386a349-2fad-4a2c-88e0-45c997cf3e6b-sys-fs\") pod \"aws-ebs-csi-driver-node-97g4h\" (UID: \"9386a349-2fad-4a2c-88e0-45c997cf3e6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97g4h"
Jul 07 11:10:52.716348 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.716323    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/58b3f62f-e190-4a7c-9a48-c7329e02c146-etc-tuned\") pod \"tuned-gb274\" (UID: \"58b3f62f-e190-4a7c-9a48-c7329e02c146\") " pod="openshift-cluster-node-tuning-operator/tuned-gb274"
Jul 07 11:10:52.716657 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.716355    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-host-kubelet\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.716657 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.716384    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-systemd-units\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.716657 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.716441    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-host-cni-bin\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.716657 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.716475    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-ovnkube-script-lib\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.716657 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.716486    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rqrgc"
Jul 07 11:10:52.716657 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.716514    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7b99b\" (UniqueName: \"kubernetes.io/projected/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-kube-api-access-7b99b\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.716657 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.716549    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/58b3f62f-e190-4a7c-9a48-c7329e02c146-etc-sysctl-d\") pod \"tuned-gb274\" (UID: \"58b3f62f-e190-4a7c-9a48-c7329e02c146\") " pod="openshift-cluster-node-tuning-operator/tuned-gb274"
Jul 07 11:10:52.716657 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.716576    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/58b3f62f-e190-4a7c-9a48-c7329e02c146-sys\") pod \"tuned-gb274\" (UID: \"58b3f62f-e190-4a7c-9a48-c7329e02c146\") " pod="openshift-cluster-node-tuning-operator/tuned-gb274"
Jul 07 11:10:52.716657 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.716600    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-var-lib-openvswitch\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.716657 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.716623    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-host-cni-netd\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.716657 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.716647    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/38a7b9ab-6691-41f8-a3b7-e47cec808b30-metrics-certs\") pod \"network-metrics-daemon-m7hkl\" (UID: \"38a7b9ab-6691-41f8-a3b7-e47cec808b30\") " pod="openshift-multus/network-metrics-daemon-m7hkl"
Jul 07 11:10:52.717084 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.716673    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/58b3f62f-e190-4a7c-9a48-c7329e02c146-etc-modprobe-d\") pod \"tuned-gb274\" (UID: \"58b3f62f-e190-4a7c-9a48-c7329e02c146\") " pod="openshift-cluster-node-tuning-operator/tuned-gb274"
Jul 07 11:10:52.717084 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.716721    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/58b3f62f-e190-4a7c-9a48-c7329e02c146-etc-sysconfig\") pod \"tuned-gb274\" (UID: \"58b3f62f-e190-4a7c-9a48-c7329e02c146\") " pod="openshift-cluster-node-tuning-operator/tuned-gb274"
Jul 07 11:10:52.717084 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.716756    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/58b3f62f-e190-4a7c-9a48-c7329e02c146-etc-sysctl-conf\") pod \"tuned-gb274\" (UID: \"58b3f62f-e190-4a7c-9a48-c7329e02c146\") " pod="openshift-cluster-node-tuning-operator/tuned-gb274"
Jul 07 11:10:52.717084 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.716801    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/58b3f62f-e190-4a7c-9a48-c7329e02c146-lib-modules\") pod \"tuned-gb274\" (UID: \"58b3f62f-e190-4a7c-9a48-c7329e02c146\") " pod="openshift-cluster-node-tuning-operator/tuned-gb274"
Jul 07 11:10:52.717084 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.716829    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/58b3f62f-e190-4a7c-9a48-c7329e02c146-host\") pod \"tuned-gb274\" (UID: \"58b3f62f-e190-4a7c-9a48-c7329e02c146\") " pod="openshift-cluster-node-tuning-operator/tuned-gb274"
Jul 07 11:10:52.717084 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.716854    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-run-ovn\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.717084 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.716920    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-host-run-ovn-kubernetes\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.717084 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.716948    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2827544c-22cd-4454-86a0-fc57d09ea878-host\") pod \"node-ca-x8c7r\" (UID: \"2827544c-22cd-4454-86a0-fc57d09ea878\") " pod="openshift-image-registry/node-ca-x8c7r"
Jul 07 11:10:52.717084 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.717005    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jptp\" (UniqueName: \"kubernetes.io/projected/2827544c-22cd-4454-86a0-fc57d09ea878-kube-api-access-8jptp\") pod \"node-ca-x8c7r\" (UID: \"2827544c-22cd-4454-86a0-fc57d09ea878\") " pod="openshift-image-registry/node-ca-x8c7r"
Jul 07 11:10:52.717084 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.717034    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q88sw\" (UniqueName: \"kubernetes.io/projected/9386a349-2fad-4a2c-88e0-45c997cf3e6b-kube-api-access-q88sw\") pod \"aws-ebs-csi-driver-node-97g4h\" (UID: \"9386a349-2fad-4a2c-88e0-45c997cf3e6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97g4h"
Jul 07 11:10:52.717084 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.717058    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/58b3f62f-e190-4a7c-9a48-c7329e02c146-etc-kubernetes\") pod \"tuned-gb274\" (UID: \"58b3f62f-e190-4a7c-9a48-c7329e02c146\") " pod="openshift-cluster-node-tuning-operator/tuned-gb274"
Jul 07 11:10:52.717562 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.717095    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-ovnkube-config\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.717562 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.717125    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzbj4\" (UniqueName: \"kubernetes.io/projected/38a7b9ab-6691-41f8-a3b7-e47cec808b30-kube-api-access-nzbj4\") pod \"network-metrics-daemon-m7hkl\" (UID: \"38a7b9ab-6691-41f8-a3b7-e47cec808b30\") " pod="openshift-multus/network-metrics-daemon-m7hkl"
Jul 07 11:10:52.717562 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.717151    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2827544c-22cd-4454-86a0-fc57d09ea878-serviceca\") pod \"node-ca-x8c7r\" (UID: \"2827544c-22cd-4454-86a0-fc57d09ea878\") " pod="openshift-image-registry/node-ca-x8c7r"
Jul 07 11:10:52.720133 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.718970    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-px74n"
Jul 07 11:10:52.720133 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:52.719077    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-px74n" podUID="a89cad11-0010-4ab5-bc82-04a3cd7a3e7b"
Jul 07 11:10:52.721485 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.721463    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 07 11:10:52.722431 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.722408    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bg78n"
Jul 07 11:10:52.722921 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.722900    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-25crw\""
Jul 07 11:10:52.723058 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.723037    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 07 11:10:52.723142 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.722907    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-8fnm4\""
Jul 07 11:10:52.723209 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.723034    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 07 11:10:52.723554 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.723529    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 07 11:10:52.723554 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.723540    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 07 11:10:52.723709 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.723670    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 07 11:10:52.723838 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.723817    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-k9xpx\""
Jul 07 11:10:52.723957 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.723925    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 07 11:10:52.728724 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.727209    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-k672b\""
Jul 07 11:10:52.728724 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.727292    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 07 11:10:52.746638 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.744732    2570 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 07 11:10:52.799338 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.799300    2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-5d7pp"
Jul 07 11:10:52.803219 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.803198    2570 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 07 11:10:52.815213 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.815173    2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-5d7pp"
Jul 07 11:10:52.817682 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.817648    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/be1eabc6-86f2-4d4f-b598-186f91dcaf4f-iptables-alerter-script\") pod \"iptables-alerter-x6xq9\" (UID: \"be1eabc6-86f2-4d4f-b598-186f91dcaf4f\") " pod="openshift-network-operator/iptables-alerter-x6xq9"
Jul 07 11:10:52.817812 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.817700    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-env-overrides\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.817812 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.817726    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-ovn-node-metrics-cert\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.817812 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.817752    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/09a382c5-57c1-4372-92e4-d24a28620c29-cnibin\") pod \"multus-additional-cni-plugins-gm84h\" (UID: \"09a382c5-57c1-4372-92e4-d24a28620c29\") " pod="openshift-multus/multus-additional-cni-plugins-gm84h"
Jul 07 11:10:52.817812 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.817775    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdkkr\" (UniqueName: \"kubernetes.io/projected/be1eabc6-86f2-4d4f-b598-186f91dcaf4f-kube-api-access-xdkkr\") pod \"iptables-alerter-x6xq9\" (UID: \"be1eabc6-86f2-4d4f-b598-186f91dcaf4f\") " pod="openshift-network-operator/iptables-alerter-x6xq9"
Jul 07 11:10:52.817812 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.817810    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/58b3f62f-e190-4a7c-9a48-c7329e02c146-run\") pod \"tuned-gb274\" (UID: \"58b3f62f-e190-4a7c-9a48-c7329e02c146\") " pod="openshift-cluster-node-tuning-operator/tuned-gb274"
Jul 07 11:10:52.818066 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.817882    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/58b3f62f-e190-4a7c-9a48-c7329e02c146-run\") pod \"tuned-gb274\" (UID: \"58b3f62f-e190-4a7c-9a48-c7329e02c146\") " pod="openshift-cluster-node-tuning-operator/tuned-gb274"
Jul 07 11:10:52.818066 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.818005    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/58b3f62f-e190-4a7c-9a48-c7329e02c146-tmp\") pod \"tuned-gb274\" (UID: \"58b3f62f-e190-4a7c-9a48-c7329e02c146\") " pod="openshift-cluster-node-tuning-operator/tuned-gb274"
Jul 07 11:10:52.818066 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.818048    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-etc-openvswitch\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.818209 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.818081    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/09a382c5-57c1-4372-92e4-d24a28620c29-system-cni-dir\") pod \"multus-additional-cni-plugins-gm84h\" (UID: \"09a382c5-57c1-4372-92e4-d24a28620c29\") " pod="openshift-multus/multus-additional-cni-plugins-gm84h"
Jul 07 11:10:52.818209 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.818109    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/09a382c5-57c1-4372-92e4-d24a28620c29-cni-binary-copy\") pod \"multus-additional-cni-plugins-gm84h\" (UID: \"09a382c5-57c1-4372-92e4-d24a28620c29\") " pod="openshift-multus/multus-additional-cni-plugins-gm84h"
Jul 07 11:10:52.818209 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.818134    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7aac7900-132e-4310-90ba-d33718fd0daa-multus-socket-dir-parent\") pod \"multus-bg78n\" (UID: \"7aac7900-132e-4310-90ba-d33718fd0daa\") " pod="openshift-multus/multus-bg78n"
Jul 07 11:10:52.818209 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.818138    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-etc-openvswitch\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.818209 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.818152    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9386a349-2fad-4a2c-88e0-45c997cf3e6b-sys-fs\") pod \"aws-ebs-csi-driver-node-97g4h\" (UID: \"9386a349-2fad-4a2c-88e0-45c997cf3e6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97g4h"
Jul 07 11:10:52.818555 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.818186    2570 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 07 11:10:52.818555 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.818219    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9386a349-2fad-4a2c-88e0-45c997cf3e6b-sys-fs\") pod \"aws-ebs-csi-driver-node-97g4h\" (UID: \"9386a349-2fad-4a2c-88e0-45c997cf3e6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97g4h"
Jul 07 11:10:52.818555 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.818215    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/58b3f62f-e190-4a7c-9a48-c7329e02c146-etc-tuned\") pod \"tuned-gb274\" (UID: \"58b3f62f-e190-4a7c-9a48-c7329e02c146\") " pod="openshift-cluster-node-tuning-operator/tuned-gb274"
Jul 07 11:10:52.818555 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.818275    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-host-cni-bin\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.818555 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.818304    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-ovnkube-script-lib\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.818555 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.818334    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/09a382c5-57c1-4372-92e4-d24a28620c29-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gm84h\" (UID: \"09a382c5-57c1-4372-92e4-d24a28620c29\") " pod="openshift-multus/multus-additional-cni-plugins-gm84h"
Jul 07 11:10:52.818555 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.818362    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-host-cni-bin\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.818555 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.818387    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7aac7900-132e-4310-90ba-d33718fd0daa-multus-cni-dir\") pod \"multus-bg78n\" (UID: \"7aac7900-132e-4310-90ba-d33718fd0daa\") " pod="openshift-multus/multus-bg78n"
Jul 07 11:10:52.818555 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.818414    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7aac7900-132e-4310-90ba-d33718fd0daa-host-var-lib-cni-bin\") pod \"multus-bg78n\" (UID: \"7aac7900-132e-4310-90ba-d33718fd0daa\") " pod="openshift-multus/multus-bg78n"
Jul 07 11:10:52.818555 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.818437    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7aac7900-132e-4310-90ba-d33718fd0daa-multus-daemon-config\") pod \"multus-bg78n\" (UID: \"7aac7900-132e-4310-90ba-d33718fd0daa\") " pod="openshift-multus/multus-bg78n"
Jul 07 11:10:52.818555 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.818462    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/be1eabc6-86f2-4d4f-b598-186f91dcaf4f-host-slash\") pod \"iptables-alerter-x6xq9\" (UID: \"be1eabc6-86f2-4d4f-b598-186f91dcaf4f\") " pod="openshift-network-operator/iptables-alerter-x6xq9"
Jul 07 11:10:52.818555 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.818487    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7aac7900-132e-4310-90ba-d33718fd0daa-system-cni-dir\") pod \"multus-bg78n\" (UID: \"7aac7900-132e-4310-90ba-d33718fd0daa\") " pod="openshift-multus/multus-bg78n"
Jul 07 11:10:52.818555 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.818512    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7aac7900-132e-4310-90ba-d33718fd0daa-etc-kubernetes\") pod \"multus-bg78n\" (UID: \"7aac7900-132e-4310-90ba-d33718fd0daa\") " pod="openshift-multus/multus-bg78n"
Jul 07 11:10:52.818555 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.818538    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqgt7\" (UniqueName: \"kubernetes.io/projected/7aac7900-132e-4310-90ba-d33718fd0daa-kube-api-access-fqgt7\") pod \"multus-bg78n\" (UID: \"7aac7900-132e-4310-90ba-d33718fd0daa\") " pod="openshift-multus/multus-bg78n"
Jul 07 11:10:52.819160 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.818571    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9gcp\" (UniqueName: \"kubernetes.io/projected/4ea618ab-a367-4d71-b0bd-e82e15e1f080-kube-api-access-r9gcp\") pod \"node-resolver-rqrgc\" (UID: \"4ea618ab-a367-4d71-b0bd-e82e15e1f080\") " pod="openshift-dns/node-resolver-rqrgc"
Jul 07 11:10:52.819160 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.818603    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/58b3f62f-e190-4a7c-9a48-c7329e02c146-etc-sysctl-conf\") pod \"tuned-gb274\" (UID: \"58b3f62f-e190-4a7c-9a48-c7329e02c146\") " pod="openshift-cluster-node-tuning-operator/tuned-gb274"
Jul 07 11:10:52.819160 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.818629    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/58b3f62f-e190-4a7c-9a48-c7329e02c146-host\") pod \"tuned-gb274\" (UID: \"58b3f62f-e190-4a7c-9a48-c7329e02c146\") " pod="openshift-cluster-node-tuning-operator/tuned-gb274"
Jul 07 11:10:52.819160 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.818653    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-host-run-ovn-kubernetes\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.819160 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.818693    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7aac7900-132e-4310-90ba-d33718fd0daa-host-var-lib-cni-multus\") pod \"multus-bg78n\" (UID: \"7aac7900-132e-4310-90ba-d33718fd0daa\") " pod="openshift-multus/multus-bg78n"
Jul 07 11:10:52.819160 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.818720    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2827544c-22cd-4454-86a0-fc57d09ea878-host\") pod \"node-ca-x8c7r\" (UID: \"2827544c-22cd-4454-86a0-fc57d09ea878\") " pod="openshift-image-registry/node-ca-x8c7r"
Jul 07 11:10:52.819160 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.818742    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-ovnkube-config\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.819160 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.818763    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a89cad11-0010-4ab5-bc82-04a3cd7a3e7b-dbus\") pod \"global-pull-secret-syncer-px74n\" (UID: \"a89cad11-0010-4ab5-bc82-04a3cd7a3e7b\") " pod="kube-system/global-pull-secret-syncer-px74n"
Jul 07 11:10:52.819160 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.818784    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c20b78c4-cef9-4f47-ac6e-0ac4303eb857-konnectivity-ca\") pod \"konnectivity-agent-dxztq\" (UID: \"c20b78c4-cef9-4f47-ac6e-0ac4303eb857\") " pod="kube-system/konnectivity-agent-dxztq"
Jul 07 11:10:52.819160 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.818807    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9386a349-2fad-4a2c-88e0-45c997cf3e6b-socket-dir\") pod \"aws-ebs-csi-driver-node-97g4h\" (UID: \"9386a349-2fad-4a2c-88e0-45c997cf3e6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97g4h"
Jul 07 11:10:52.819160 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.818828    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-run-systemd\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.819160 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.818851    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vng8\" (UniqueName: \"kubernetes.io/projected/09a382c5-57c1-4372-92e4-d24a28620c29-kube-api-access-6vng8\") pod \"multus-additional-cni-plugins-gm84h\" (UID: \"09a382c5-57c1-4372-92e4-d24a28620c29\") " pod="openshift-multus/multus-additional-cni-plugins-gm84h"
Jul 07 11:10:52.819160 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.818874    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7aac7900-132e-4310-90ba-d33718fd0daa-host-var-lib-kubelet\") pod \"multus-bg78n\" (UID: \"7aac7900-132e-4310-90ba-d33718fd0daa\") " pod="openshift-multus/multus-bg78n"
Jul 07 11:10:52.819160 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.818894    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4ea618ab-a367-4d71-b0bd-e82e15e1f080-hosts-file\") pod \"node-resolver-rqrgc\" (UID: \"4ea618ab-a367-4d71-b0bd-e82e15e1f080\") " pod="openshift-dns/node-resolver-rqrgc"
Jul 07 11:10:52.819160 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.818919    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9386a349-2fad-4a2c-88e0-45c997cf3e6b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-97g4h\" (UID: \"9386a349-2fad-4a2c-88e0-45c997cf3e6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97g4h"
Jul 07 11:10:52.819160 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.818941    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9386a349-2fad-4a2c-88e0-45c997cf3e6b-device-dir\") pod \"aws-ebs-csi-driver-node-97g4h\" (UID: \"9386a349-2fad-4a2c-88e0-45c997cf3e6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97g4h"
Jul 07 11:10:52.819160 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.818975    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-run-openvswitch\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.819841 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.818997    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-log-socket\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.819841 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.819021    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.819841 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.819046    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/09a382c5-57c1-4372-92e4-d24a28620c29-os-release\") pod \"multus-additional-cni-plugins-gm84h\" (UID: \"09a382c5-57c1-4372-92e4-d24a28620c29\") " pod="openshift-multus/multus-additional-cni-plugins-gm84h"
Jul 07 11:10:52.819841 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.819071    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7aac7900-132e-4310-90ba-d33718fd0daa-os-release\") pod \"multus-bg78n\" (UID: \"7aac7900-132e-4310-90ba-d33718fd0daa\") " pod="openshift-multus/multus-bg78n"
Jul 07 11:10:52.819841 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.819097    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-node-log\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.819841 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.819171    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-ovnkube-script-lib\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.819841 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.819243    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7aac7900-132e-4310-90ba-d33718fd0daa-cni-binary-copy\") pod \"multus-bg78n\" (UID: \"7aac7900-132e-4310-90ba-d33718fd0daa\") " pod="openshift-multus/multus-bg78n"
Jul 07 11:10:52.819841 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.819349    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9386a349-2fad-4a2c-88e0-45c997cf3e6b-registration-dir\") pod \"aws-ebs-csi-driver-node-97g4h\" (UID: \"9386a349-2fad-4a2c-88e0-45c997cf3e6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97g4h"
Jul 07 11:10:52.819841 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.819375    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9386a349-2fad-4a2c-88e0-45c997cf3e6b-etc-selinux\") pod \"aws-ebs-csi-driver-node-97g4h\" (UID: \"9386a349-2fad-4a2c-88e0-45c997cf3e6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97g4h"
Jul 07 11:10:52.819841 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.819420    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/58b3f62f-e190-4a7c-9a48-c7329e02c146-etc-systemd\") pod \"tuned-gb274\" (UID: \"58b3f62f-e190-4a7c-9a48-c7329e02c146\") " pod="openshift-cluster-node-tuning-operator/tuned-gb274"
Jul 07 11:10:52.819841 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.819445    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bk9f2\" (UniqueName: \"kubernetes.io/projected/58b3f62f-e190-4a7c-9a48-c7329e02c146-kube-api-access-bk9f2\") pod \"tuned-gb274\" (UID: \"58b3f62f-e190-4a7c-9a48-c7329e02c146\") " pod="openshift-cluster-node-tuning-operator/tuned-gb274"
Jul 07 11:10:52.819841 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.819470    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7aac7900-132e-4310-90ba-d33718fd0daa-cnibin\") pod \"multus-bg78n\" (UID: \"7aac7900-132e-4310-90ba-d33718fd0daa\") " pod="openshift-multus/multus-bg78n"
Jul 07 11:10:52.819841 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.819503    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9386a349-2fad-4a2c-88e0-45c997cf3e6b-socket-dir\") pod \"aws-ebs-csi-driver-node-97g4h\" (UID: \"9386a349-2fad-4a2c-88e0-45c997cf3e6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97g4h"
Jul 07 11:10:52.819841 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.819508    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7aac7900-132e-4310-90ba-d33718fd0daa-host-run-multus-certs\") pod \"multus-bg78n\" (UID: \"7aac7900-132e-4310-90ba-d33718fd0daa\") " pod="openshift-multus/multus-bg78n"
Jul 07 11:10:52.819841 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.819537    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-host-kubelet\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.819841 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.819557    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-systemd-units\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.819841 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.819572    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-run-systemd\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.820467 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.819575    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7b99b\" (UniqueName: \"kubernetes.io/projected/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-kube-api-access-7b99b\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.820467 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.819615    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a89cad11-0010-4ab5-bc82-04a3cd7a3e7b-kubelet-config\") pod \"global-pull-secret-syncer-px74n\" (UID: \"a89cad11-0010-4ab5-bc82-04a3cd7a3e7b\") " pod="kube-system/global-pull-secret-syncer-px74n"
Jul 07 11:10:52.820467 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.819642    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7aac7900-132e-4310-90ba-d33718fd0daa-host-run-k8s-cni-cncf-io\") pod \"multus-bg78n\" (UID: \"7aac7900-132e-4310-90ba-d33718fd0daa\") " pod="openshift-multus/multus-bg78n"
Jul 07 11:10:52.820467 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.819674    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/58b3f62f-e190-4a7c-9a48-c7329e02c146-etc-sysctl-d\") pod \"tuned-gb274\" (UID: \"58b3f62f-e190-4a7c-9a48-c7329e02c146\") " pod="openshift-cluster-node-tuning-operator/tuned-gb274"
Jul 07 11:10:52.820467 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.819700    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/58b3f62f-e190-4a7c-9a48-c7329e02c146-sys\") pod \"tuned-gb274\" (UID: \"58b3f62f-e190-4a7c-9a48-c7329e02c146\") " pod="openshift-cluster-node-tuning-operator/tuned-gb274"
Jul 07 11:10:52.820467 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.819746    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-var-lib-openvswitch\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.820467 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.819782    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-host-cni-netd\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.820467 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.819975    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-host-run-ovn-kubernetes\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.820467 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.820001    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7aac7900-132e-4310-90ba-d33718fd0daa-hostroot\") pod \"multus-bg78n\" (UID: \"7aac7900-132e-4310-90ba-d33718fd0daa\") " pod="openshift-multus/multus-bg78n"
Jul 07 11:10:52.820467 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.820019    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7aac7900-132e-4310-90ba-d33718fd0daa-multus-conf-dir\") pod \"multus-bg78n\" (UID: \"7aac7900-132e-4310-90ba-d33718fd0daa\") " pod="openshift-multus/multus-bg78n"
Jul 07 11:10:52.820467 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.820050    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kbzb\" (UniqueName: \"kubernetes.io/projected/f971d15b-e001-45d9-ba6f-e0b60efc55cc-kube-api-access-4kbzb\") pod \"network-check-target-n5mhp\" (UID: \"f971d15b-e001-45d9-ba6f-e0b60efc55cc\") " pod="openshift-network-diagnostics/network-check-target-n5mhp"
Jul 07 11:10:52.820467 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.820057    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/58b3f62f-e190-4a7c-9a48-c7329e02c146-etc-sysctl-conf\") pod \"tuned-gb274\" (UID: \"58b3f62f-e190-4a7c-9a48-c7329e02c146\") " pod="openshift-cluster-node-tuning-operator/tuned-gb274"
Jul 07 11:10:52.820467 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.820072    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/38a7b9ab-6691-41f8-a3b7-e47cec808b30-metrics-certs\") pod \"network-metrics-daemon-m7hkl\" (UID: \"38a7b9ab-6691-41f8-a3b7-e47cec808b30\") " pod="openshift-multus/network-metrics-daemon-m7hkl"
Jul 07 11:10:52.820467 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.820092    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/58b3f62f-e190-4a7c-9a48-c7329e02c146-etc-modprobe-d\") pod \"tuned-gb274\" (UID: \"58b3f62f-e190-4a7c-9a48-c7329e02c146\") " pod="openshift-cluster-node-tuning-operator/tuned-gb274"
Jul 07 11:10:52.820467 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.820106    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/58b3f62f-e190-4a7c-9a48-c7329e02c146-host\") pod \"tuned-gb274\" (UID: \"58b3f62f-e190-4a7c-9a48-c7329e02c146\") " pod="openshift-cluster-node-tuning-operator/tuned-gb274"
Jul 07 11:10:52.820467 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.820110    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/58b3f62f-e190-4a7c-9a48-c7329e02c146-etc-sysconfig\") pod \"tuned-gb274\" (UID: \"58b3f62f-e190-4a7c-9a48-c7329e02c146\") " pod="openshift-cluster-node-tuning-operator/tuned-gb274"
Jul 07 11:10:52.820467 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.820145    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/58b3f62f-e190-4a7c-9a48-c7329e02c146-etc-sysconfig\") pod \"tuned-gb274\" (UID: \"58b3f62f-e190-4a7c-9a48-c7329e02c146\") " pod="openshift-cluster-node-tuning-operator/tuned-gb274"
Jul 07 11:10:52.821010 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.820145    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/58b3f62f-e190-4a7c-9a48-c7329e02c146-lib-modules\") pod \"tuned-gb274\" (UID: \"58b3f62f-e190-4a7c-9a48-c7329e02c146\") " pod="openshift-cluster-node-tuning-operator/tuned-gb274"
Jul 07 11:10:52.821010 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.820170    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2827544c-22cd-4454-86a0-fc57d09ea878-host\") pod \"node-ca-x8c7r\" (UID: \"2827544c-22cd-4454-86a0-fc57d09ea878\") " pod="openshift-image-registry/node-ca-x8c7r"
Jul 07 11:10:52.821010 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.820180    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-run-ovn\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.821010 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.820192    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-host-cni-netd\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.821010 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.820209    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/09a382c5-57c1-4372-92e4-d24a28620c29-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gm84h\" (UID: \"09a382c5-57c1-4372-92e4-d24a28620c29\") " pod="openshift-multus/multus-additional-cni-plugins-gm84h"
Jul 07 11:10:52.821010 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.820229    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7aac7900-132e-4310-90ba-d33718fd0daa-host-run-netns\") pod \"multus-bg78n\" (UID: \"7aac7900-132e-4310-90ba-d33718fd0daa\") " pod="openshift-multus/multus-bg78n"
Jul 07 11:10:52.821010 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.820232    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-run-ovn\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.821010 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.820247    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jptp\" (UniqueName: \"kubernetes.io/projected/2827544c-22cd-4454-86a0-fc57d09ea878-kube-api-access-8jptp\") pod \"node-ca-x8c7r\" (UID: \"2827544c-22cd-4454-86a0-fc57d09ea878\") " pod="openshift-image-registry/node-ca-x8c7r"
Jul 07 11:10:52.821010 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.820267    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q88sw\" (UniqueName: \"kubernetes.io/projected/9386a349-2fad-4a2c-88e0-45c997cf3e6b-kube-api-access-q88sw\") pod \"aws-ebs-csi-driver-node-97g4h\" (UID: \"9386a349-2fad-4a2c-88e0-45c997cf3e6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97g4h"
Jul 07 11:10:52.821010 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.820284    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/58b3f62f-e190-4a7c-9a48-c7329e02c146-etc-kubernetes\") pod \"tuned-gb274\" (UID: \"58b3f62f-e190-4a7c-9a48-c7329e02c146\") " pod="openshift-cluster-node-tuning-operator/tuned-gb274"
Jul 07 11:10:52.821010 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.820301    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzbj4\" (UniqueName: \"kubernetes.io/projected/38a7b9ab-6691-41f8-a3b7-e47cec808b30-kube-api-access-nzbj4\") pod \"network-metrics-daemon-m7hkl\" (UID: \"38a7b9ab-6691-41f8-a3b7-e47cec808b30\") " pod="openshift-multus/network-metrics-daemon-m7hkl"
Jul 07 11:10:52.821010 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.820318    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2827544c-22cd-4454-86a0-fc57d09ea878-serviceca\") pod \"node-ca-x8c7r\" (UID: \"2827544c-22cd-4454-86a0-fc57d09ea878\") " pod="openshift-image-registry/node-ca-x8c7r"
Jul 07 11:10:52.821010 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.820323    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/58b3f62f-e190-4a7c-9a48-c7329e02c146-lib-modules\") pod \"tuned-gb274\" (UID: \"58b3f62f-e190-4a7c-9a48-c7329e02c146\") " pod="openshift-cluster-node-tuning-operator/tuned-gb274"
Jul 07 11:10:52.821010 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.820342    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/58b3f62f-e190-4a7c-9a48-c7329e02c146-var-lib-kubelet\") pod \"tuned-gb274\" (UID: \"58b3f62f-e190-4a7c-9a48-c7329e02c146\") " pod="openshift-cluster-node-tuning-operator/tuned-gb274"
Jul 07 11:10:52.821010 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.820360    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-host-run-netns\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.821010 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.820378    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/09a382c5-57c1-4372-92e4-d24a28620c29-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gm84h\" (UID: \"09a382c5-57c1-4372-92e4-d24a28620c29\") " pod="openshift-multus/multus-additional-cni-plugins-gm84h"
Jul 07 11:10:52.821010 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.820395    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c20b78c4-cef9-4f47-ac6e-0ac4303eb857-agent-certs\") pod \"konnectivity-agent-dxztq\" (UID: \"c20b78c4-cef9-4f47-ac6e-0ac4303eb857\") " pod="kube-system/konnectivity-agent-dxztq"
Jul 07 11:10:52.821701 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.820431    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ea618ab-a367-4d71-b0bd-e82e15e1f080-tmp-dir\") pod \"node-resolver-rqrgc\" (UID: \"4ea618ab-a367-4d71-b0bd-e82e15e1f080\") " pod="openshift-dns/node-resolver-rqrgc"
Jul 07 11:10:52.821701 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.820451    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-host-slash\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.821701 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.820469    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a89cad11-0010-4ab5-bc82-04a3cd7a3e7b-original-pull-secret\") pod \"global-pull-secret-syncer-px74n\" (UID: \"a89cad11-0010-4ab5-bc82-04a3cd7a3e7b\") " pod="kube-system/global-pull-secret-syncer-px74n"
Jul 07 11:10:52.821701 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.820540    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/58b3f62f-e190-4a7c-9a48-c7329e02c146-sys\") pod \"tuned-gb274\" (UID: \"58b3f62f-e190-4a7c-9a48-c7329e02c146\") " pod="openshift-cluster-node-tuning-operator/tuned-gb274"
Jul 07 11:10:52.821701 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.820548    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-host-kubelet\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.821701 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.820564    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-var-lib-openvswitch\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.821701 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.820590    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-systemd-units\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.821701 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:52.820634    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 11:10:52.821701 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:52.820692    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38a7b9ab-6691-41f8-a3b7-e47cec808b30-metrics-certs podName:38a7b9ab-6691-41f8-a3b7-e47cec808b30 nodeName:}" failed. No retries permitted until 2026-07-07 11:10:53.320667162 +0000 UTC m=+3.030563528 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/38a7b9ab-6691-41f8-a3b7-e47cec808b30-metrics-certs") pod "network-metrics-daemon-m7hkl" (UID: "38a7b9ab-6691-41f8-a3b7-e47cec808b30") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 11:10:52.821701 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.820872    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/58b3f62f-e190-4a7c-9a48-c7329e02c146-etc-modprobe-d\") pod \"tuned-gb274\" (UID: \"58b3f62f-e190-4a7c-9a48-c7329e02c146\") " pod="openshift-cluster-node-tuning-operator/tuned-gb274"
Jul 07 11:10:52.821701 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.821102    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-ovnkube-config\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.821701 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.821171    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/58b3f62f-e190-4a7c-9a48-c7329e02c146-var-lib-kubelet\") pod \"tuned-gb274\" (UID: \"58b3f62f-e190-4a7c-9a48-c7329e02c146\") " pod="openshift-cluster-node-tuning-operator/tuned-gb274"
Jul 07 11:10:52.821701 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.821178    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/58b3f62f-e190-4a7c-9a48-c7329e02c146-etc-kubernetes\") pod \"tuned-gb274\" (UID: \"58b3f62f-e190-4a7c-9a48-c7329e02c146\") " pod="openshift-cluster-node-tuning-operator/tuned-gb274"
Jul 07 11:10:52.821701 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.821204    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-host-run-netns\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.821701 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.821268    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-host-slash\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.821701 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.821449    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9386a349-2fad-4a2c-88e0-45c997cf3e6b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-97g4h\" (UID: \"9386a349-2fad-4a2c-88e0-45c997cf3e6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97g4h"
Jul 07 11:10:52.821701 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.821492    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-node-log\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.822448 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.821498    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9386a349-2fad-4a2c-88e0-45c997cf3e6b-registration-dir\") pod \"aws-ebs-csi-driver-node-97g4h\" (UID: \"9386a349-2fad-4a2c-88e0-45c997cf3e6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97g4h"
Jul 07 11:10:52.822448 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.821542    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9386a349-2fad-4a2c-88e0-45c997cf3e6b-device-dir\") pod \"aws-ebs-csi-driver-node-97g4h\" (UID: \"9386a349-2fad-4a2c-88e0-45c997cf3e6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97g4h"
Jul 07 11:10:52.822448 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.821583    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-run-openvswitch\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.822448 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.821593    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9386a349-2fad-4a2c-88e0-45c997cf3e6b-etc-selinux\") pod \"aws-ebs-csi-driver-node-97g4h\" (UID: \"9386a349-2fad-4a2c-88e0-45c997cf3e6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97g4h"
Jul 07 11:10:52.822448 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.821618    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-log-socket\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.822448 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.821627    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/58b3f62f-e190-4a7c-9a48-c7329e02c146-etc-systemd\") pod \"tuned-gb274\" (UID: \"58b3f62f-e190-4a7c-9a48-c7329e02c146\") " pod="openshift-cluster-node-tuning-operator/tuned-gb274"
Jul 07 11:10:52.822448 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.821652    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.822448 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.821925    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/58b3f62f-e190-4a7c-9a48-c7329e02c146-tmp\") pod \"tuned-gb274\" (UID: \"58b3f62f-e190-4a7c-9a48-c7329e02c146\") " pod="openshift-cluster-node-tuning-operator/tuned-gb274"
Jul 07 11:10:52.822448 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.821928    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/58b3f62f-e190-4a7c-9a48-c7329e02c146-etc-sysctl-d\") pod \"tuned-gb274\" (UID: \"58b3f62f-e190-4a7c-9a48-c7329e02c146\") " pod="openshift-cluster-node-tuning-operator/tuned-gb274"
Jul 07 11:10:52.822448 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.822353    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2827544c-22cd-4454-86a0-fc57d09ea878-serviceca\") pod \"node-ca-x8c7r\" (UID: \"2827544c-22cd-4454-86a0-fc57d09ea878\") " pod="openshift-image-registry/node-ca-x8c7r"
Jul 07 11:10:52.822448 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.822418    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-39.ec2.internal" event={"ID":"7d767ad34fcd7349932e3bffab974a69","Type":"ContainerStarted","Data":"e851dcb1c8565c109a36436f499ca0f8f29e5708d3b7e24afe85d10d510d5a39"}
Jul 07 11:10:52.823018 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.822625    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-ovn-node-metrics-cert\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.823018 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.822973    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-env-overrides\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.823285 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.823257    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-39.ec2.internal" event={"ID":"53f76728005a66da1fcc175f0398436c","Type":"ContainerStarted","Data":"c9f177c5ca73098d06ec72fabcafd53fddefde66e539195cf9c5d357bcfe20ba"}
Jul 07 11:10:52.824236 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.824218    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/58b3f62f-e190-4a7c-9a48-c7329e02c146-etc-tuned\") pod \"tuned-gb274\" (UID: \"58b3f62f-e190-4a7c-9a48-c7329e02c146\") " pod="openshift-cluster-node-tuning-operator/tuned-gb274"
Jul 07 11:10:52.847119 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.847065    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bk9f2\" (UniqueName: \"kubernetes.io/projected/58b3f62f-e190-4a7c-9a48-c7329e02c146-kube-api-access-bk9f2\") pod \"tuned-gb274\" (UID: \"58b3f62f-e190-4a7c-9a48-c7329e02c146\") " pod="openshift-cluster-node-tuning-operator/tuned-gb274"
Jul 07 11:10:52.847268 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.847088    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7b99b\" (UniqueName: \"kubernetes.io/projected/ce823451-93f9-4fe7-8ebc-f78efdd21ca2-kube-api-access-7b99b\") pod \"ovnkube-node-dbsg9\" (UID: \"ce823451-93f9-4fe7-8ebc-f78efdd21ca2\") " pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:52.848249 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.848222    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q88sw\" (UniqueName: \"kubernetes.io/projected/9386a349-2fad-4a2c-88e0-45c997cf3e6b-kube-api-access-q88sw\") pod \"aws-ebs-csi-driver-node-97g4h\" (UID: \"9386a349-2fad-4a2c-88e0-45c997cf3e6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97g4h"
Jul 07 11:10:52.849717 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.849685    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzbj4\" (UniqueName: \"kubernetes.io/projected/38a7b9ab-6691-41f8-a3b7-e47cec808b30-kube-api-access-nzbj4\") pod \"network-metrics-daemon-m7hkl\" (UID: \"38a7b9ab-6691-41f8-a3b7-e47cec808b30\") " pod="openshift-multus/network-metrics-daemon-m7hkl"
Jul 07 11:10:52.855887 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.855859    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jptp\" (UniqueName: \"kubernetes.io/projected/2827544c-22cd-4454-86a0-fc57d09ea878-kube-api-access-8jptp\") pod \"node-ca-x8c7r\" (UID: \"2827544c-22cd-4454-86a0-fc57d09ea878\") " pod="openshift-image-registry/node-ca-x8c7r"
Jul 07 11:10:52.893577 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.893542    2570 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 11:10:52.920784 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.920746    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a89cad11-0010-4ab5-bc82-04a3cd7a3e7b-kubelet-config\") pod \"global-pull-secret-syncer-px74n\" (UID: \"a89cad11-0010-4ab5-bc82-04a3cd7a3e7b\") " pod="kube-system/global-pull-secret-syncer-px74n"
Jul 07 11:10:52.920784 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.920790    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7aac7900-132e-4310-90ba-d33718fd0daa-host-run-k8s-cni-cncf-io\") pod \"multus-bg78n\" (UID: \"7aac7900-132e-4310-90ba-d33718fd0daa\") " pod="openshift-multus/multus-bg78n"
Jul 07 11:10:52.921032 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.920815    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7aac7900-132e-4310-90ba-d33718fd0daa-hostroot\") pod \"multus-bg78n\" (UID: \"7aac7900-132e-4310-90ba-d33718fd0daa\") " pod="openshift-multus/multus-bg78n"
Jul 07 11:10:52.921032 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.920843    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7aac7900-132e-4310-90ba-d33718fd0daa-multus-conf-dir\") pod \"multus-bg78n\" (UID: \"7aac7900-132e-4310-90ba-d33718fd0daa\") " pod="openshift-multus/multus-bg78n"
Jul 07 11:10:52.921032 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.920862    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a89cad11-0010-4ab5-bc82-04a3cd7a3e7b-kubelet-config\") pod \"global-pull-secret-syncer-px74n\" (UID: \"a89cad11-0010-4ab5-bc82-04a3cd7a3e7b\") " pod="kube-system/global-pull-secret-syncer-px74n"
Jul 07 11:10:52.921032 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.920881    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kbzb\" (UniqueName: \"kubernetes.io/projected/f971d15b-e001-45d9-ba6f-e0b60efc55cc-kube-api-access-4kbzb\") pod \"network-check-target-n5mhp\" (UID: \"f971d15b-e001-45d9-ba6f-e0b60efc55cc\") " pod="openshift-network-diagnostics/network-check-target-n5mhp"
Jul 07 11:10:52.921032 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.920887    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7aac7900-132e-4310-90ba-d33718fd0daa-host-run-k8s-cni-cncf-io\") pod \"multus-bg78n\" (UID: \"7aac7900-132e-4310-90ba-d33718fd0daa\") " pod="openshift-multus/multus-bg78n"
Jul 07 11:10:52.921032 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.920944    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7aac7900-132e-4310-90ba-d33718fd0daa-hostroot\") pod \"multus-bg78n\" (UID: \"7aac7900-132e-4310-90ba-d33718fd0daa\") " pod="openshift-multus/multus-bg78n"
Jul 07 11:10:52.921032 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.920947    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/09a382c5-57c1-4372-92e4-d24a28620c29-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gm84h\" (UID: \"09a382c5-57c1-4372-92e4-d24a28620c29\") " pod="openshift-multus/multus-additional-cni-plugins-gm84h"
Jul 07 11:10:52.921032 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.920982    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7aac7900-132e-4310-90ba-d33718fd0daa-multus-conf-dir\") pod \"multus-bg78n\" (UID: \"7aac7900-132e-4310-90ba-d33718fd0daa\") " pod="openshift-multus/multus-bg78n"
Jul 07 11:10:52.921032 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.921018    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7aac7900-132e-4310-90ba-d33718fd0daa-host-run-netns\") pod \"multus-bg78n\" (UID: \"7aac7900-132e-4310-90ba-d33718fd0daa\") " pod="openshift-multus/multus-bg78n"
Jul 07 11:10:52.921510 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.921059    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/09a382c5-57c1-4372-92e4-d24a28620c29-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gm84h\" (UID: \"09a382c5-57c1-4372-92e4-d24a28620c29\") " pod="openshift-multus/multus-additional-cni-plugins-gm84h"
Jul 07 11:10:52.921510 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.921085    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c20b78c4-cef9-4f47-ac6e-0ac4303eb857-agent-certs\") pod \"konnectivity-agent-dxztq\" (UID: \"c20b78c4-cef9-4f47-ac6e-0ac4303eb857\") " pod="kube-system/konnectivity-agent-dxztq"
Jul 07 11:10:52.921510 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.921111    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ea618ab-a367-4d71-b0bd-e82e15e1f080-tmp-dir\") pod \"node-resolver-rqrgc\" (UID: \"4ea618ab-a367-4d71-b0bd-e82e15e1f080\") " pod="openshift-dns/node-resolver-rqrgc"
Jul 07 11:10:52.921510 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.921110    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7aac7900-132e-4310-90ba-d33718fd0daa-host-run-netns\") pod \"multus-bg78n\" (UID: \"7aac7900-132e-4310-90ba-d33718fd0daa\") " pod="openshift-multus/multus-bg78n"
Jul 07 11:10:52.921510 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.921174    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a89cad11-0010-4ab5-bc82-04a3cd7a3e7b-original-pull-secret\") pod \"global-pull-secret-syncer-px74n\" (UID: \"a89cad11-0010-4ab5-bc82-04a3cd7a3e7b\") " pod="kube-system/global-pull-secret-syncer-px74n"
Jul 07 11:10:52.921510 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.921227    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/be1eabc6-86f2-4d4f-b598-186f91dcaf4f-iptables-alerter-script\") pod \"iptables-alerter-x6xq9\" (UID: \"be1eabc6-86f2-4d4f-b598-186f91dcaf4f\") " pod="openshift-network-operator/iptables-alerter-x6xq9"
Jul 07 11:10:52.921510 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.921256    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/09a382c5-57c1-4372-92e4-d24a28620c29-cnibin\") pod \"multus-additional-cni-plugins-gm84h\" (UID: \"09a382c5-57c1-4372-92e4-d24a28620c29\") " pod="openshift-multus/multus-additional-cni-plugins-gm84h"
Jul 07 11:10:52.921510 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.921283    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdkkr\" (UniqueName: \"kubernetes.io/projected/be1eabc6-86f2-4d4f-b598-186f91dcaf4f-kube-api-access-xdkkr\") pod \"iptables-alerter-x6xq9\" (UID: \"be1eabc6-86f2-4d4f-b598-186f91dcaf4f\") " pod="openshift-network-operator/iptables-alerter-x6xq9"
Jul 07 11:10:52.921510 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.921295    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/09a382c5-57c1-4372-92e4-d24a28620c29-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gm84h\" (UID: \"09a382c5-57c1-4372-92e4-d24a28620c29\") " pod="openshift-multus/multus-additional-cni-plugins-gm84h"
Jul 07 11:10:52.921510 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:52.921343    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 11:10:52.921510 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.921386    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/09a382c5-57c1-4372-92e4-d24a28620c29-cnibin\") pod \"multus-additional-cni-plugins-gm84h\" (UID: \"09a382c5-57c1-4372-92e4-d24a28620c29\") " pod="openshift-multus/multus-additional-cni-plugins-gm84h"
Jul 07 11:10:52.921510 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:52.921426    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a89cad11-0010-4ab5-bc82-04a3cd7a3e7b-original-pull-secret podName:a89cad11-0010-4ab5-bc82-04a3cd7a3e7b nodeName:}" failed. No retries permitted until 2026-07-07 11:10:53.421393973 +0000 UTC m=+3.131290358 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a89cad11-0010-4ab5-bc82-04a3cd7a3e7b-original-pull-secret") pod "global-pull-secret-syncer-px74n" (UID: "a89cad11-0010-4ab5-bc82-04a3cd7a3e7b") : object "kube-system"/"original-pull-secret" not registered
Jul 07 11:10:52.921510 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.921514    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/09a382c5-57c1-4372-92e4-d24a28620c29-system-cni-dir\") pod \"multus-additional-cni-plugins-gm84h\" (UID: \"09a382c5-57c1-4372-92e4-d24a28620c29\") " pod="openshift-multus/multus-additional-cni-plugins-gm84h"
Jul 07 11:10:52.922166 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.921547    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/09a382c5-57c1-4372-92e4-d24a28620c29-cni-binary-copy\") pod \"multus-additional-cni-plugins-gm84h\" (UID: \"09a382c5-57c1-4372-92e4-d24a28620c29\") " pod="openshift-multus/multus-additional-cni-plugins-gm84h"
Jul 07 11:10:52.922166 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.921579    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7aac7900-132e-4310-90ba-d33718fd0daa-multus-socket-dir-parent\") pod \"multus-bg78n\" (UID: \"7aac7900-132e-4310-90ba-d33718fd0daa\") " pod="openshift-multus/multus-bg78n"
Jul 07 11:10:52.922166 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.921579    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/09a382c5-57c1-4372-92e4-d24a28620c29-system-cni-dir\") pod \"multus-additional-cni-plugins-gm84h\" (UID: \"09a382c5-57c1-4372-92e4-d24a28620c29\") " pod="openshift-multus/multus-additional-cni-plugins-gm84h"
Jul 07 11:10:52.922166 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.921592    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ea618ab-a367-4d71-b0bd-e82e15e1f080-tmp-dir\") pod \"node-resolver-rqrgc\" (UID: \"4ea618ab-a367-4d71-b0bd-e82e15e1f080\") " pod="openshift-dns/node-resolver-rqrgc"
Jul 07 11:10:52.922166 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.921612    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/09a382c5-57c1-4372-92e4-d24a28620c29-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gm84h\" (UID: \"09a382c5-57c1-4372-92e4-d24a28620c29\") " pod="openshift-multus/multus-additional-cni-plugins-gm84h"
Jul 07 11:10:52.922166 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.921677    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7aac7900-132e-4310-90ba-d33718fd0daa-multus-cni-dir\") pod \"multus-bg78n\" (UID: \"7aac7900-132e-4310-90ba-d33718fd0daa\") " pod="openshift-multus/multus-bg78n"
Jul 07 11:10:52.922166 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.921684    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7aac7900-132e-4310-90ba-d33718fd0daa-multus-socket-dir-parent\") pod \"multus-bg78n\" (UID: \"7aac7900-132e-4310-90ba-d33718fd0daa\") " pod="openshift-multus/multus-bg78n"
Jul 07 11:10:52.922166 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.921705    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7aac7900-132e-4310-90ba-d33718fd0daa-host-var-lib-cni-bin\") pod \"multus-bg78n\" (UID: \"7aac7900-132e-4310-90ba-d33718fd0daa\") " pod="openshift-multus/multus-bg78n"
Jul 07 11:10:52.922166 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.921730    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7aac7900-132e-4310-90ba-d33718fd0daa-multus-daemon-config\") pod \"multus-bg78n\" (UID: \"7aac7900-132e-4310-90ba-d33718fd0daa\") " pod="openshift-multus/multus-bg78n"
Jul 07 11:10:52.922166 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.921757    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/be1eabc6-86f2-4d4f-b598-186f91dcaf4f-host-slash\") pod \"iptables-alerter-x6xq9\" (UID: \"be1eabc6-86f2-4d4f-b598-186f91dcaf4f\") " pod="openshift-network-operator/iptables-alerter-x6xq9"
Jul 07 11:10:52.922166 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.921764    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7aac7900-132e-4310-90ba-d33718fd0daa-multus-cni-dir\") pod \"multus-bg78n\" (UID: \"7aac7900-132e-4310-90ba-d33718fd0daa\") " pod="openshift-multus/multus-bg78n"
Jul 07 11:10:52.922166 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.921782    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7aac7900-132e-4310-90ba-d33718fd0daa-system-cni-dir\") pod \"multus-bg78n\" (UID: \"7aac7900-132e-4310-90ba-d33718fd0daa\") " pod="openshift-multus/multus-bg78n"
Jul 07 11:10:52.922166 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.921796    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/09a382c5-57c1-4372-92e4-d24a28620c29-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gm84h\" (UID: \"09a382c5-57c1-4372-92e4-d24a28620c29\") " pod="openshift-multus/multus-additional-cni-plugins-gm84h"
Jul 07 11:10:52.922166 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.921809    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7aac7900-132e-4310-90ba-d33718fd0daa-etc-kubernetes\") pod \"multus-bg78n\" (UID: \"7aac7900-132e-4310-90ba-d33718fd0daa\") " pod="openshift-multus/multus-bg78n"
Jul 07 11:10:52.922166 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.921849    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqgt7\" (UniqueName: \"kubernetes.io/projected/7aac7900-132e-4310-90ba-d33718fd0daa-kube-api-access-fqgt7\") pod \"multus-bg78n\" (UID: \"7aac7900-132e-4310-90ba-d33718fd0daa\") " pod="openshift-multus/multus-bg78n"
Jul 07 11:10:52.922166 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.921885    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/be1eabc6-86f2-4d4f-b598-186f91dcaf4f-host-slash\") pod \"iptables-alerter-x6xq9\" (UID: \"be1eabc6-86f2-4d4f-b598-186f91dcaf4f\") " pod="openshift-network-operator/iptables-alerter-x6xq9"
Jul 07 11:10:52.922166 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.921879    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9gcp\" (UniqueName: \"kubernetes.io/projected/4ea618ab-a367-4d71-b0bd-e82e15e1f080-kube-api-access-r9gcp\") pod \"node-resolver-rqrgc\" (UID: \"4ea618ab-a367-4d71-b0bd-e82e15e1f080\") " pod="openshift-dns/node-resolver-rqrgc"
Jul 07 11:10:52.922962 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.921912    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/be1eabc6-86f2-4d4f-b598-186f91dcaf4f-iptables-alerter-script\") pod \"iptables-alerter-x6xq9\" (UID: \"be1eabc6-86f2-4d4f-b598-186f91dcaf4f\") " pod="openshift-network-operator/iptables-alerter-x6xq9"
Jul 07 11:10:52.922962 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.921853    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7aac7900-132e-4310-90ba-d33718fd0daa-etc-kubernetes\") pod \"multus-bg78n\" (UID: \"7aac7900-132e-4310-90ba-d33718fd0daa\") " pod="openshift-multus/multus-bg78n"
Jul 07 11:10:52.922962 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.921936    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7aac7900-132e-4310-90ba-d33718fd0daa-host-var-lib-cni-multus\") pod \"multus-bg78n\" (UID: \"7aac7900-132e-4310-90ba-d33718fd0daa\") " pod="openshift-multus/multus-bg78n"
Jul 07 11:10:52.922962 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.921946    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7aac7900-132e-4310-90ba-d33718fd0daa-system-cni-dir\") pod \"multus-bg78n\" (UID: \"7aac7900-132e-4310-90ba-d33718fd0daa\") " pod="openshift-multus/multus-bg78n"
Jul 07 11:10:52.922962 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.921974    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a89cad11-0010-4ab5-bc82-04a3cd7a3e7b-dbus\") pod \"global-pull-secret-syncer-px74n\" (UID: \"a89cad11-0010-4ab5-bc82-04a3cd7a3e7b\") " pod="kube-system/global-pull-secret-syncer-px74n"
Jul 07 11:10:52.922962 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.921989    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7aac7900-132e-4310-90ba-d33718fd0daa-host-var-lib-cni-bin\") pod \"multus-bg78n\" (UID: \"7aac7900-132e-4310-90ba-d33718fd0daa\") " pod="openshift-multus/multus-bg78n"
Jul 07 11:10:52.922962 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.922003    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c20b78c4-cef9-4f47-ac6e-0ac4303eb857-konnectivity-ca\") pod \"konnectivity-agent-dxztq\" (UID: \"c20b78c4-cef9-4f47-ac6e-0ac4303eb857\") " pod="kube-system/konnectivity-agent-dxztq"
Jul 07 11:10:52.922962 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.922030    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7aac7900-132e-4310-90ba-d33718fd0daa-host-var-lib-cni-multus\") pod \"multus-bg78n\" (UID: \"7aac7900-132e-4310-90ba-d33718fd0daa\") " pod="openshift-multus/multus-bg78n"
Jul 07 11:10:52.922962 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.922032    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vng8\" (UniqueName: \"kubernetes.io/projected/09a382c5-57c1-4372-92e4-d24a28620c29-kube-api-access-6vng8\") pod \"multus-additional-cni-plugins-gm84h\" (UID: \"09a382c5-57c1-4372-92e4-d24a28620c29\") " pod="openshift-multus/multus-additional-cni-plugins-gm84h"
Jul 07 11:10:52.922962 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.922100    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/09a382c5-57c1-4372-92e4-d24a28620c29-cni-binary-copy\") pod \"multus-additional-cni-plugins-gm84h\" (UID: \"09a382c5-57c1-4372-92e4-d24a28620c29\") " pod="openshift-multus/multus-additional-cni-plugins-gm84h"
Jul 07 11:10:52.922962 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.922201    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7aac7900-132e-4310-90ba-d33718fd0daa-host-var-lib-kubelet\") pod \"multus-bg78n\" (UID: \"7aac7900-132e-4310-90ba-d33718fd0daa\") " pod="openshift-multus/multus-bg78n"
Jul 07 11:10:52.922962 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.922233    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4ea618ab-a367-4d71-b0bd-e82e15e1f080-hosts-file\") pod \"node-resolver-rqrgc\" (UID: \"4ea618ab-a367-4d71-b0bd-e82e15e1f080\") " pod="openshift-dns/node-resolver-rqrgc"
Jul 07 11:10:52.922962 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.922249    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a89cad11-0010-4ab5-bc82-04a3cd7a3e7b-dbus\") pod \"global-pull-secret-syncer-px74n\" (UID: \"a89cad11-0010-4ab5-bc82-04a3cd7a3e7b\") " pod="kube-system/global-pull-secret-syncer-px74n"
Jul 07 11:10:52.922962 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.922279    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7aac7900-132e-4310-90ba-d33718fd0daa-host-var-lib-kubelet\") pod \"multus-bg78n\" (UID: \"7aac7900-132e-4310-90ba-d33718fd0daa\") " pod="openshift-multus/multus-bg78n"
Jul 07 11:10:52.922962 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.922294    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4ea618ab-a367-4d71-b0bd-e82e15e1f080-hosts-file\") pod \"node-resolver-rqrgc\" (UID: \"4ea618ab-a367-4d71-b0bd-e82e15e1f080\") " pod="openshift-dns/node-resolver-rqrgc"
Jul 07 11:10:52.922962 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.922307    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/09a382c5-57c1-4372-92e4-d24a28620c29-os-release\") pod \"multus-additional-cni-plugins-gm84h\" (UID: \"09a382c5-57c1-4372-92e4-d24a28620c29\") " pod="openshift-multus/multus-additional-cni-plugins-gm84h"
Jul 07 11:10:52.922962 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.922325    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7aac7900-132e-4310-90ba-d33718fd0daa-os-release\") pod \"multus-bg78n\" (UID: \"7aac7900-132e-4310-90ba-d33718fd0daa\") " pod="openshift-multus/multus-bg78n"
Jul 07 11:10:52.922962 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.922343    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7aac7900-132e-4310-90ba-d33718fd0daa-cni-binary-copy\") pod \"multus-bg78n\" (UID: \"7aac7900-132e-4310-90ba-d33718fd0daa\") " pod="openshift-multus/multus-bg78n"
Jul 07 11:10:52.923667 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.922358    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7aac7900-132e-4310-90ba-d33718fd0daa-multus-daemon-config\") pod \"multus-bg78n\" (UID: \"7aac7900-132e-4310-90ba-d33718fd0daa\") " pod="openshift-multus/multus-bg78n"
Jul 07 11:10:52.923667 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.922363    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/09a382c5-57c1-4372-92e4-d24a28620c29-os-release\") pod \"multus-additional-cni-plugins-gm84h\" (UID: \"09a382c5-57c1-4372-92e4-d24a28620c29\") " pod="openshift-multus/multus-additional-cni-plugins-gm84h"
Jul 07 11:10:52.923667 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.922436    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7aac7900-132e-4310-90ba-d33718fd0daa-os-release\") pod \"multus-bg78n\" (UID: \"7aac7900-132e-4310-90ba-d33718fd0daa\") " pod="openshift-multus/multus-bg78n"
Jul 07 11:10:52.923667 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.922437    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/09a382c5-57c1-4372-92e4-d24a28620c29-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gm84h\" (UID: \"09a382c5-57c1-4372-92e4-d24a28620c29\") " pod="openshift-multus/multus-additional-cni-plugins-gm84h"
Jul 07 11:10:52.923667 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.922468    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7aac7900-132e-4310-90ba-d33718fd0daa-cnibin\") pod \"multus-bg78n\" (UID: \"7aac7900-132e-4310-90ba-d33718fd0daa\") " pod="openshift-multus/multus-bg78n"
Jul 07 11:10:52.923667 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.922512    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7aac7900-132e-4310-90ba-d33718fd0daa-host-run-multus-certs\") pod \"multus-bg78n\" (UID: \"7aac7900-132e-4310-90ba-d33718fd0daa\") " pod="openshift-multus/multus-bg78n"
Jul 07 11:10:52.923667 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.922606    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7aac7900-132e-4310-90ba-d33718fd0daa-cnibin\") pod \"multus-bg78n\" (UID: \"7aac7900-132e-4310-90ba-d33718fd0daa\") " pod="openshift-multus/multus-bg78n"
Jul 07 11:10:52.923667 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.922648    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c20b78c4-cef9-4f47-ac6e-0ac4303eb857-konnectivity-ca\") pod \"konnectivity-agent-dxztq\" (UID: \"c20b78c4-cef9-4f47-ac6e-0ac4303eb857\") " pod="kube-system/konnectivity-agent-dxztq"
Jul 07 11:10:52.923667 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.922638    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7aac7900-132e-4310-90ba-d33718fd0daa-host-run-multus-certs\") pod \"multus-bg78n\" (UID: \"7aac7900-132e-4310-90ba-d33718fd0daa\") " pod="openshift-multus/multus-bg78n"
Jul 07 11:10:52.923667 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.922788    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7aac7900-132e-4310-90ba-d33718fd0daa-cni-binary-copy\") pod \"multus-bg78n\" (UID: \"7aac7900-132e-4310-90ba-d33718fd0daa\") " pod="openshift-multus/multus-bg78n"
Jul 07 11:10:52.924225 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.924198    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c20b78c4-cef9-4f47-ac6e-0ac4303eb857-agent-certs\") pod \"konnectivity-agent-dxztq\" (UID: \"c20b78c4-cef9-4f47-ac6e-0ac4303eb857\") " pod="kube-system/konnectivity-agent-dxztq"
Jul 07 11:10:52.929754 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:52.929676    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 11:10:52.929754 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:52.929703    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 11:10:52.929754 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:52.929719    2570 projected.go:196] Error preparing data for projected volume kube-api-access-4kbzb for pod openshift-network-diagnostics/network-check-target-n5mhp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 11:10:52.930160 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:52.929791    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f971d15b-e001-45d9-ba6f-e0b60efc55cc-kube-api-access-4kbzb podName:f971d15b-e001-45d9-ba6f-e0b60efc55cc nodeName:}" failed. No retries permitted until 2026-07-07 11:10:53.429773585 +0000 UTC m=+3.139669974 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-4kbzb" (UniqueName: "kubernetes.io/projected/f971d15b-e001-45d9-ba6f-e0b60efc55cc-kube-api-access-4kbzb") pod "network-check-target-n5mhp" (UID: "f971d15b-e001-45d9-ba6f-e0b60efc55cc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 11:10:52.932202 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.932177    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdkkr\" (UniqueName: \"kubernetes.io/projected/be1eabc6-86f2-4d4f-b598-186f91dcaf4f-kube-api-access-xdkkr\") pod \"iptables-alerter-x6xq9\" (UID: \"be1eabc6-86f2-4d4f-b598-186f91dcaf4f\") " pod="openshift-network-operator/iptables-alerter-x6xq9"
Jul 07 11:10:52.933166 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.933139    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqgt7\" (UniqueName: \"kubernetes.io/projected/7aac7900-132e-4310-90ba-d33718fd0daa-kube-api-access-fqgt7\") pod \"multus-bg78n\" (UID: \"7aac7900-132e-4310-90ba-d33718fd0daa\") " pod="openshift-multus/multus-bg78n"
Jul 07 11:10:52.933928 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.933883    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9gcp\" (UniqueName: \"kubernetes.io/projected/4ea618ab-a367-4d71-b0bd-e82e15e1f080-kube-api-access-r9gcp\") pod \"node-resolver-rqrgc\" (UID: \"4ea618ab-a367-4d71-b0bd-e82e15e1f080\") " pod="openshift-dns/node-resolver-rqrgc"
Jul 07 11:10:52.934041 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:52.934023    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vng8\" (UniqueName: \"kubernetes.io/projected/09a382c5-57c1-4372-92e4-d24a28620c29-kube-api-access-6vng8\") pod \"multus-additional-cni-plugins-gm84h\" (UID: \"09a382c5-57c1-4372-92e4-d24a28620c29\") " pod="openshift-multus/multus-additional-cni-plugins-gm84h"
Jul 07 11:10:53.004932 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:53.004889    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-x8c7r"
Jul 07 11:10:53.015606 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:53.015577    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97g4h"
Jul 07 11:10:53.026016 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:53.025991    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gb274"
Jul 07 11:10:53.033090 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:53.033059    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:10:53.040576 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:53.040555    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gm84h"
Jul 07 11:10:53.052956 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:53.052913    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dxztq"
Jul 07 11:10:53.062870 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:53.062838    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-x6xq9"
Jul 07 11:10:53.069956 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:53.069933    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rqrgc"
Jul 07 11:10:53.077881 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:53.077856    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bg78n"
Jul 07 11:10:53.326255 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:53.326166    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/38a7b9ab-6691-41f8-a3b7-e47cec808b30-metrics-certs\") pod \"network-metrics-daemon-m7hkl\" (UID: \"38a7b9ab-6691-41f8-a3b7-e47cec808b30\") " pod="openshift-multus/network-metrics-daemon-m7hkl"
Jul 07 11:10:53.326445 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:53.326322    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 11:10:53.326445 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:53.326414    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38a7b9ab-6691-41f8-a3b7-e47cec808b30-metrics-certs podName:38a7b9ab-6691-41f8-a3b7-e47cec808b30 nodeName:}" failed. No retries permitted until 2026-07-07 11:10:54.326396995 +0000 UTC m=+4.036293362 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/38a7b9ab-6691-41f8-a3b7-e47cec808b30-metrics-certs") pod "network-metrics-daemon-m7hkl" (UID: "38a7b9ab-6691-41f8-a3b7-e47cec808b30") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 11:10:53.427297 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:53.427248    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a89cad11-0010-4ab5-bc82-04a3cd7a3e7b-original-pull-secret\") pod \"global-pull-secret-syncer-px74n\" (UID: \"a89cad11-0010-4ab5-bc82-04a3cd7a3e7b\") " pod="kube-system/global-pull-secret-syncer-px74n"
Jul 07 11:10:53.427496 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:53.427467    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 11:10:53.427573 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:53.427558    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a89cad11-0010-4ab5-bc82-04a3cd7a3e7b-original-pull-secret podName:a89cad11-0010-4ab5-bc82-04a3cd7a3e7b nodeName:}" failed. No retries permitted until 2026-07-07 11:10:54.427535882 +0000 UTC m=+4.137432264 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a89cad11-0010-4ab5-bc82-04a3cd7a3e7b-original-pull-secret") pod "global-pull-secret-syncer-px74n" (UID: "a89cad11-0010-4ab5-bc82-04a3cd7a3e7b") : object "kube-system"/"original-pull-secret" not registered
Jul 07 11:10:53.527890 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:53.527846    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kbzb\" (UniqueName: \"kubernetes.io/projected/f971d15b-e001-45d9-ba6f-e0b60efc55cc-kube-api-access-4kbzb\") pod \"network-check-target-n5mhp\" (UID: \"f971d15b-e001-45d9-ba6f-e0b60efc55cc\") " pod="openshift-network-diagnostics/network-check-target-n5mhp"
Jul 07 11:10:53.528101 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:53.528024    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 11:10:53.528101 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:53.528043    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 11:10:53.528101 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:53.528054    2570 projected.go:196] Error preparing data for projected volume kube-api-access-4kbzb for pod openshift-network-diagnostics/network-check-target-n5mhp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 11:10:53.528280 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:53.528125    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f971d15b-e001-45d9-ba6f-e0b60efc55cc-kube-api-access-4kbzb podName:f971d15b-e001-45d9-ba6f-e0b60efc55cc nodeName:}" failed. No retries permitted until 2026-07-07 11:10:54.528107177 +0000 UTC m=+4.238003544 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-4kbzb" (UniqueName: "kubernetes.io/projected/f971d15b-e001-45d9-ba6f-e0b60efc55cc-kube-api-access-4kbzb") pod "network-check-target-n5mhp" (UID: "f971d15b-e001-45d9-ba6f-e0b60efc55cc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 11:10:53.814894 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:53.814810    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5mhp"
Jul 07 11:10:53.815251 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:53.814982    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n5mhp" podUID="f971d15b-e001-45d9-ba6f-e0b60efc55cc"
Jul 07 11:10:53.822869 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:53.822833    2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 11:05:52 +0000 UTC" deadline="2028-03-28 21:36:00.239235474 +0000 UTC"
Jul 07 11:10:53.822869 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:53.822866    2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15130h25m6.416373309s"
Jul 07 11:10:53.953696 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:53.953654    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2827544c_22cd_4454_86a0_fc57d09ea878.slice/crio-e140f07d38650409bea0925cda2f6df2f21802dc706145a66317eaa10c65cd76 WatchSource:0}: Error finding container e140f07d38650409bea0925cda2f6df2f21802dc706145a66317eaa10c65cd76: Status 404 returned error can't find the container with id e140f07d38650409bea0925cda2f6df2f21802dc706145a66317eaa10c65cd76
Jul 07 11:10:53.955477 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:53.955446    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe1eabc6_86f2_4d4f_b598_186f91dcaf4f.slice/crio-f3b0f4d82d58aebbfd60ed0e70ec3aa0390009c6d9a10969816c40d8020e96ad WatchSource:0}: Error finding container f3b0f4d82d58aebbfd60ed0e70ec3aa0390009c6d9a10969816c40d8020e96ad: Status 404 returned error can't find the container with id f3b0f4d82d58aebbfd60ed0e70ec3aa0390009c6d9a10969816c40d8020e96ad
Jul 07 11:10:53.958114 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:53.958093    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9386a349_2fad_4a2c_88e0_45c997cf3e6b.slice/crio-15280c05fe53b515634df004548e5a79fca77b16f63151ab5e58ff8c153d7bd9 WatchSource:0}: Error finding container 15280c05fe53b515634df004548e5a79fca77b16f63151ab5e58ff8c153d7bd9: Status 404 returned error can't find the container with id 15280c05fe53b515634df004548e5a79fca77b16f63151ab5e58ff8c153d7bd9
Jul 07 11:10:53.962073 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:53.961947    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce823451_93f9_4fe7_8ebc_f78efdd21ca2.slice/crio-f0fd331011dcbb260a9287532a99aa961782149372380ca281e6cfd0b0cb0ff2 WatchSource:0}: Error finding container f0fd331011dcbb260a9287532a99aa961782149372380ca281e6cfd0b0cb0ff2: Status 404 returned error can't find the container with id f0fd331011dcbb260a9287532a99aa961782149372380ca281e6cfd0b0cb0ff2
Jul 07 11:10:53.965297 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:53.965137    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09a382c5_57c1_4372_92e4_d24a28620c29.slice/crio-74c9b5e9d41272e3fc4608312c7d3277c6cda2a4fc0dc7ff806d011f8673edf6 WatchSource:0}: Error finding container 74c9b5e9d41272e3fc4608312c7d3277c6cda2a4fc0dc7ff806d011f8673edf6: Status 404 returned error can't find the container with id 74c9b5e9d41272e3fc4608312c7d3277c6cda2a4fc0dc7ff806d011f8673edf6
Jul 07 11:10:53.990527 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:53.990499    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7aac7900_132e_4310_90ba_d33718fd0daa.slice/crio-e9518f0c89943ab5871be487a32bed70a535a33e91f475808ab61b6673cc1e9f WatchSource:0}: Error finding container e9518f0c89943ab5871be487a32bed70a535a33e91f475808ab61b6673cc1e9f: Status 404 returned error can't find the container with id e9518f0c89943ab5871be487a32bed70a535a33e91f475808ab61b6673cc1e9f
Jul 07 11:10:53.991717 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:10:53.991603    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ea618ab_a367_4d71_b0bd_e82e15e1f080.slice/crio-baf7e0c354f0b3e3bdeab663e1929f64c91c0010005a0258bf895ae64778f88d WatchSource:0}: Error finding container baf7e0c354f0b3e3bdeab663e1929f64c91c0010005a0258bf895ae64778f88d: Status 404 returned error can't find the container with id baf7e0c354f0b3e3bdeab663e1929f64c91c0010005a0258bf895ae64778f88d
Jul 07 11:10:54.335007 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:54.334976    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/38a7b9ab-6691-41f8-a3b7-e47cec808b30-metrics-certs\") pod \"network-metrics-daemon-m7hkl\" (UID: \"38a7b9ab-6691-41f8-a3b7-e47cec808b30\") " pod="openshift-multus/network-metrics-daemon-m7hkl"
Jul 07 11:10:54.335155 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:54.335122    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 11:10:54.335216 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:54.335200    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38a7b9ab-6691-41f8-a3b7-e47cec808b30-metrics-certs podName:38a7b9ab-6691-41f8-a3b7-e47cec808b30 nodeName:}" failed. No retries permitted until 2026-07-07 11:10:56.33517742 +0000 UTC m=+6.045073804 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/38a7b9ab-6691-41f8-a3b7-e47cec808b30-metrics-certs") pod "network-metrics-daemon-m7hkl" (UID: "38a7b9ab-6691-41f8-a3b7-e47cec808b30") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 11:10:54.436598 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:54.435916    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a89cad11-0010-4ab5-bc82-04a3cd7a3e7b-original-pull-secret\") pod \"global-pull-secret-syncer-px74n\" (UID: \"a89cad11-0010-4ab5-bc82-04a3cd7a3e7b\") " pod="kube-system/global-pull-secret-syncer-px74n"
Jul 07 11:10:54.436598 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:54.436133    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 11:10:54.436598 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:54.436206    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a89cad11-0010-4ab5-bc82-04a3cd7a3e7b-original-pull-secret podName:a89cad11-0010-4ab5-bc82-04a3cd7a3e7b nodeName:}" failed. No retries permitted until 2026-07-07 11:10:56.436187049 +0000 UTC m=+6.146083419 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a89cad11-0010-4ab5-bc82-04a3cd7a3e7b-original-pull-secret") pod "global-pull-secret-syncer-px74n" (UID: "a89cad11-0010-4ab5-bc82-04a3cd7a3e7b") : object "kube-system"/"original-pull-secret" not registered
Jul 07 11:10:54.536437 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:54.536392    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kbzb\" (UniqueName: \"kubernetes.io/projected/f971d15b-e001-45d9-ba6f-e0b60efc55cc-kube-api-access-4kbzb\") pod \"network-check-target-n5mhp\" (UID: \"f971d15b-e001-45d9-ba6f-e0b60efc55cc\") " pod="openshift-network-diagnostics/network-check-target-n5mhp"
Jul 07 11:10:54.536660 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:54.536629    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 11:10:54.536660 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:54.536655    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 11:10:54.536794 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:54.536668    2570 projected.go:196] Error preparing data for projected volume kube-api-access-4kbzb for pod openshift-network-diagnostics/network-check-target-n5mhp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 11:10:54.536794 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:54.536734    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f971d15b-e001-45d9-ba6f-e0b60efc55cc-kube-api-access-4kbzb podName:f971d15b-e001-45d9-ba6f-e0b60efc55cc nodeName:}" failed. No retries permitted until 2026-07-07 11:10:56.536713417 +0000 UTC m=+6.246609808 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-4kbzb" (UniqueName: "kubernetes.io/projected/f971d15b-e001-45d9-ba6f-e0b60efc55cc-kube-api-access-4kbzb") pod "network-check-target-n5mhp" (UID: "f971d15b-e001-45d9-ba6f-e0b60efc55cc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 11:10:54.817479 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:54.817378    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m7hkl"
Jul 07 11:10:54.817943 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:54.817543    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m7hkl" podUID="38a7b9ab-6691-41f8-a3b7-e47cec808b30"
Jul 07 11:10:54.817943 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:54.817707    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-px74n"
Jul 07 11:10:54.817943 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:54.817857    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-px74n" podUID="a89cad11-0010-4ab5-bc82-04a3cd7a3e7b"
Jul 07 11:10:54.823263 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:54.823199    2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 11:05:52 +0000 UTC" deadline="2028-03-09 20:51:04.515659462 +0000 UTC"
Jul 07 11:10:54.823263 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:54.823262    2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14673h40m9.692401777s"
Jul 07 11:10:54.846939 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:54.846901    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gm84h" event={"ID":"09a382c5-57c1-4372-92e4-d24a28620c29","Type":"ContainerStarted","Data":"74c9b5e9d41272e3fc4608312c7d3277c6cda2a4fc0dc7ff806d011f8673edf6"}
Jul 07 11:10:54.850923 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:54.850803    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9" event={"ID":"ce823451-93f9-4fe7-8ebc-f78efdd21ca2","Type":"ContainerStarted","Data":"f0fd331011dcbb260a9287532a99aa961782149372380ca281e6cfd0b0cb0ff2"}
Jul 07 11:10:54.852624 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:54.852550    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97g4h" event={"ID":"9386a349-2fad-4a2c-88e0-45c997cf3e6b","Type":"ContainerStarted","Data":"15280c05fe53b515634df004548e5a79fca77b16f63151ab5e58ff8c153d7bd9"}
Jul 07 11:10:54.857218 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:54.857181    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-x6xq9" event={"ID":"be1eabc6-86f2-4d4f-b598-186f91dcaf4f","Type":"ContainerStarted","Data":"f3b0f4d82d58aebbfd60ed0e70ec3aa0390009c6d9a10969816c40d8020e96ad"}
Jul 07 11:10:54.867352 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:54.867312    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-39.ec2.internal" event={"ID":"7d767ad34fcd7349932e3bffab974a69","Type":"ContainerStarted","Data":"e7da2486ce1e0e2764dead6b96242ba731d17d0369a7755fbd94fd2b440a1cf6"}
Jul 07 11:10:54.872171 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:54.872135    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bg78n" event={"ID":"7aac7900-132e-4310-90ba-d33718fd0daa","Type":"ContainerStarted","Data":"e9518f0c89943ab5871be487a32bed70a535a33e91f475808ab61b6673cc1e9f"}
Jul 07 11:10:54.887721 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:54.887679    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gb274" event={"ID":"58b3f62f-e190-4a7c-9a48-c7329e02c146","Type":"ContainerStarted","Data":"0485f8b44b4c697b692dd3ba3aacb8377e7db5ecd39c1d2efb79e336aa203d44"}
Jul 07 11:10:54.890163 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:54.890093    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-39.ec2.internal" podStartSLOduration=2.890073287 podStartE2EDuration="2.890073287s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:10:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:10:54.889552298 +0000 UTC m=+4.599448693" watchObservedRunningTime="2026-07-07 11:10:54.890073287 +0000 UTC m=+4.599969679"
Jul 07 11:10:54.894857 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:54.894825    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dxztq" event={"ID":"c20b78c4-cef9-4f47-ac6e-0ac4303eb857","Type":"ContainerStarted","Data":"5bd009d79d67c76c7c0d18df5f6d40d2189d63fbed0cef874e1c6d03a1b404de"}
Jul 07 11:10:54.913773 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:54.913733    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-x8c7r" event={"ID":"2827544c-22cd-4454-86a0-fc57d09ea878","Type":"ContainerStarted","Data":"e140f07d38650409bea0925cda2f6df2f21802dc706145a66317eaa10c65cd76"}
Jul 07 11:10:54.935453 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:54.935365    2570 generic.go:356] "Generic (PLEG): container finished" podID="53f76728005a66da1fcc175f0398436c" containerID="6396c72bb16f7f45da35ebd9c9ae3ba95ef8002dd291bdde3238d26083cebb94" exitCode=0
Jul 07 11:10:54.935683 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:54.935504    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-39.ec2.internal" event={"ID":"53f76728005a66da1fcc175f0398436c","Type":"ContainerDied","Data":"6396c72bb16f7f45da35ebd9c9ae3ba95ef8002dd291bdde3238d26083cebb94"}
Jul 07 11:10:54.947190 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:54.947157    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rqrgc" event={"ID":"4ea618ab-a367-4d71-b0bd-e82e15e1f080","Type":"ContainerStarted","Data":"baf7e0c354f0b3e3bdeab663e1929f64c91c0010005a0258bf895ae64778f88d"}
Jul 07 11:10:55.815138 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:55.815101    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5mhp"
Jul 07 11:10:55.815358 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:55.815241    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n5mhp" podUID="f971d15b-e001-45d9-ba6f-e0b60efc55cc"
Jul 07 11:10:55.963431 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:55.963385    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-39.ec2.internal" event={"ID":"53f76728005a66da1fcc175f0398436c","Type":"ContainerStarted","Data":"015c4f5fcd610a98743516171929e8149bdf493179979aef5b312890d7048238"}
Jul 07 11:10:55.991903 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:55.991843    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-39.ec2.internal" podStartSLOduration=3.99181915 podStartE2EDuration="3.99181915s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:10:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:10:55.991604865 +0000 UTC m=+5.701501252" watchObservedRunningTime="2026-07-07 11:10:55.99181915 +0000 UTC m=+5.701715539"
Jul 07 11:10:56.353569 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:56.352949    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/38a7b9ab-6691-41f8-a3b7-e47cec808b30-metrics-certs\") pod \"network-metrics-daemon-m7hkl\" (UID: \"38a7b9ab-6691-41f8-a3b7-e47cec808b30\") " pod="openshift-multus/network-metrics-daemon-m7hkl"
Jul 07 11:10:56.353569 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:56.353134    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 11:10:56.353569 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:56.353199    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38a7b9ab-6691-41f8-a3b7-e47cec808b30-metrics-certs podName:38a7b9ab-6691-41f8-a3b7-e47cec808b30 nodeName:}" failed. No retries permitted until 2026-07-07 11:11:00.353182029 +0000 UTC m=+10.063078396 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/38a7b9ab-6691-41f8-a3b7-e47cec808b30-metrics-certs") pod "network-metrics-daemon-m7hkl" (UID: "38a7b9ab-6691-41f8-a3b7-e47cec808b30") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 11:10:56.453556 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:56.453516    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a89cad11-0010-4ab5-bc82-04a3cd7a3e7b-original-pull-secret\") pod \"global-pull-secret-syncer-px74n\" (UID: \"a89cad11-0010-4ab5-bc82-04a3cd7a3e7b\") " pod="kube-system/global-pull-secret-syncer-px74n"
Jul 07 11:10:56.453715 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:56.453685    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 11:10:56.453788 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:56.453757    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a89cad11-0010-4ab5-bc82-04a3cd7a3e7b-original-pull-secret podName:a89cad11-0010-4ab5-bc82-04a3cd7a3e7b nodeName:}" failed. No retries permitted until 2026-07-07 11:11:00.453739745 +0000 UTC m=+10.163636126 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a89cad11-0010-4ab5-bc82-04a3cd7a3e7b-original-pull-secret") pod "global-pull-secret-syncer-px74n" (UID: "a89cad11-0010-4ab5-bc82-04a3cd7a3e7b") : object "kube-system"/"original-pull-secret" not registered
Jul 07 11:10:56.554200 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:56.554164    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kbzb\" (UniqueName: \"kubernetes.io/projected/f971d15b-e001-45d9-ba6f-e0b60efc55cc-kube-api-access-4kbzb\") pod \"network-check-target-n5mhp\" (UID: \"f971d15b-e001-45d9-ba6f-e0b60efc55cc\") " pod="openshift-network-diagnostics/network-check-target-n5mhp"
Jul 07 11:10:56.554439 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:56.554383    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 11:10:56.554439 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:56.554403    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 11:10:56.554439 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:56.554413    2570 projected.go:196] Error preparing data for projected volume kube-api-access-4kbzb for pod openshift-network-diagnostics/network-check-target-n5mhp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 11:10:56.554634 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:56.554471    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f971d15b-e001-45d9-ba6f-e0b60efc55cc-kube-api-access-4kbzb podName:f971d15b-e001-45d9-ba6f-e0b60efc55cc nodeName:}" failed. No retries permitted until 2026-07-07 11:11:00.554451027 +0000 UTC m=+10.264347415 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-4kbzb" (UniqueName: "kubernetes.io/projected/f971d15b-e001-45d9-ba6f-e0b60efc55cc-kube-api-access-4kbzb") pod "network-check-target-n5mhp" (UID: "f971d15b-e001-45d9-ba6f-e0b60efc55cc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 11:10:56.814639 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:56.814551    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m7hkl"
Jul 07 11:10:56.814639 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:56.814616    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-px74n"
Jul 07 11:10:56.814869 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:56.814747    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-px74n" podUID="a89cad11-0010-4ab5-bc82-04a3cd7a3e7b"
Jul 07 11:10:56.817160 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:56.817086    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m7hkl" podUID="38a7b9ab-6691-41f8-a3b7-e47cec808b30"
Jul 07 11:10:57.815901 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:57.815354    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5mhp"
Jul 07 11:10:57.815901 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:57.815485    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n5mhp" podUID="f971d15b-e001-45d9-ba6f-e0b60efc55cc"
Jul 07 11:10:58.815467 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:58.815436    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-px74n"
Jul 07 11:10:58.815655 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:58.815566    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-px74n" podUID="a89cad11-0010-4ab5-bc82-04a3cd7a3e7b"
Jul 07 11:10:58.817453 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:58.817430    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m7hkl"
Jul 07 11:10:58.817825 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:58.817566    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m7hkl" podUID="38a7b9ab-6691-41f8-a3b7-e47cec808b30"
Jul 07 11:10:59.814723 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:10:59.814694    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5mhp"
Jul 07 11:10:59.814918 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:10:59.814818    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n5mhp" podUID="f971d15b-e001-45d9-ba6f-e0b60efc55cc"
Jul 07 11:11:00.392823 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:00.392275    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/38a7b9ab-6691-41f8-a3b7-e47cec808b30-metrics-certs\") pod \"network-metrics-daemon-m7hkl\" (UID: \"38a7b9ab-6691-41f8-a3b7-e47cec808b30\") " pod="openshift-multus/network-metrics-daemon-m7hkl"
Jul 07 11:11:00.392823 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:00.392465    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 11:11:00.392823 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:00.392535    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38a7b9ab-6691-41f8-a3b7-e47cec808b30-metrics-certs podName:38a7b9ab-6691-41f8-a3b7-e47cec808b30 nodeName:}" failed. No retries permitted until 2026-07-07 11:11:08.392520208 +0000 UTC m=+18.102416572 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/38a7b9ab-6691-41f8-a3b7-e47cec808b30-metrics-certs") pod "network-metrics-daemon-m7hkl" (UID: "38a7b9ab-6691-41f8-a3b7-e47cec808b30") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 11:11:00.493188 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:00.493150    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a89cad11-0010-4ab5-bc82-04a3cd7a3e7b-original-pull-secret\") pod \"global-pull-secret-syncer-px74n\" (UID: \"a89cad11-0010-4ab5-bc82-04a3cd7a3e7b\") " pod="kube-system/global-pull-secret-syncer-px74n"
Jul 07 11:11:00.493403 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:00.493354    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 11:11:00.493452 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:00.493440    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a89cad11-0010-4ab5-bc82-04a3cd7a3e7b-original-pull-secret podName:a89cad11-0010-4ab5-bc82-04a3cd7a3e7b nodeName:}" failed. No retries permitted until 2026-07-07 11:11:08.493417698 +0000 UTC m=+18.203314066 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a89cad11-0010-4ab5-bc82-04a3cd7a3e7b-original-pull-secret") pod "global-pull-secret-syncer-px74n" (UID: "a89cad11-0010-4ab5-bc82-04a3cd7a3e7b") : object "kube-system"/"original-pull-secret" not registered
Jul 07 11:11:00.593774 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:00.593723    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kbzb\" (UniqueName: \"kubernetes.io/projected/f971d15b-e001-45d9-ba6f-e0b60efc55cc-kube-api-access-4kbzb\") pod \"network-check-target-n5mhp\" (UID: \"f971d15b-e001-45d9-ba6f-e0b60efc55cc\") " pod="openshift-network-diagnostics/network-check-target-n5mhp"
Jul 07 11:11:00.593931 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:00.593907    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 11:11:00.593991 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:00.593938    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 11:11:00.593991 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:00.593951    2570 projected.go:196] Error preparing data for projected volume kube-api-access-4kbzb for pod openshift-network-diagnostics/network-check-target-n5mhp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 11:11:00.594069 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:00.594028    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f971d15b-e001-45d9-ba6f-e0b60efc55cc-kube-api-access-4kbzb podName:f971d15b-e001-45d9-ba6f-e0b60efc55cc nodeName:}" failed. No retries permitted until 2026-07-07 11:11:08.594005494 +0000 UTC m=+18.303901879 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-4kbzb" (UniqueName: "kubernetes.io/projected/f971d15b-e001-45d9-ba6f-e0b60efc55cc-kube-api-access-4kbzb") pod "network-check-target-n5mhp" (UID: "f971d15b-e001-45d9-ba6f-e0b60efc55cc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 11:11:00.817096 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:00.816447    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m7hkl"
Jul 07 11:11:00.817096 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:00.816494    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-px74n"
Jul 07 11:11:00.817096 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:00.816594    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m7hkl" podUID="38a7b9ab-6691-41f8-a3b7-e47cec808b30"
Jul 07 11:11:00.817096 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:00.817050    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-px74n" podUID="a89cad11-0010-4ab5-bc82-04a3cd7a3e7b"
Jul 07 11:11:01.815013 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:01.814945    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5mhp"
Jul 07 11:11:01.815498 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:01.815158    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n5mhp" podUID="f971d15b-e001-45d9-ba6f-e0b60efc55cc"
Jul 07 11:11:02.815495 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:02.815451    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-px74n"
Jul 07 11:11:02.820380 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:02.819794    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-px74n" podUID="a89cad11-0010-4ab5-bc82-04a3cd7a3e7b"
Jul 07 11:11:02.820380 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:02.819934    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m7hkl"
Jul 07 11:11:02.820380 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:02.820173    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m7hkl" podUID="38a7b9ab-6691-41f8-a3b7-e47cec808b30"
Jul 07 11:11:03.814905 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:03.814869    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5mhp"
Jul 07 11:11:03.815065 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:03.815001    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n5mhp" podUID="f971d15b-e001-45d9-ba6f-e0b60efc55cc"
Jul 07 11:11:04.815009 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:04.814974    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-px74n"
Jul 07 11:11:04.815432 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:04.814974    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m7hkl"
Jul 07 11:11:04.815432 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:04.815102    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-px74n" podUID="a89cad11-0010-4ab5-bc82-04a3cd7a3e7b"
Jul 07 11:11:04.815432 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:04.815175    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m7hkl" podUID="38a7b9ab-6691-41f8-a3b7-e47cec808b30"
Jul 07 11:11:05.814790 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:05.814745    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5mhp"
Jul 07 11:11:05.814970 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:05.814872    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n5mhp" podUID="f971d15b-e001-45d9-ba6f-e0b60efc55cc"
Jul 07 11:11:06.815565 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:06.815535    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m7hkl"
Jul 07 11:11:06.816002 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:06.815534    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-px74n"
Jul 07 11:11:06.816002 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:06.815682    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m7hkl" podUID="38a7b9ab-6691-41f8-a3b7-e47cec808b30"
Jul 07 11:11:06.816002 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:06.815756    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-px74n" podUID="a89cad11-0010-4ab5-bc82-04a3cd7a3e7b"
Jul 07 11:11:07.815529 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:07.815490    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5mhp"
Jul 07 11:11:07.815772 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:07.815628    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n5mhp" podUID="f971d15b-e001-45d9-ba6f-e0b60efc55cc"
Jul 07 11:11:08.456494 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:08.456455    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/38a7b9ab-6691-41f8-a3b7-e47cec808b30-metrics-certs\") pod \"network-metrics-daemon-m7hkl\" (UID: \"38a7b9ab-6691-41f8-a3b7-e47cec808b30\") " pod="openshift-multus/network-metrics-daemon-m7hkl"
Jul 07 11:11:08.456687 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:08.456641    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 11:11:08.456737 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:08.456720    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38a7b9ab-6691-41f8-a3b7-e47cec808b30-metrics-certs podName:38a7b9ab-6691-41f8-a3b7-e47cec808b30 nodeName:}" failed. No retries permitted until 2026-07-07 11:11:24.456699581 +0000 UTC m=+34.166595957 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/38a7b9ab-6691-41f8-a3b7-e47cec808b30-metrics-certs") pod "network-metrics-daemon-m7hkl" (UID: "38a7b9ab-6691-41f8-a3b7-e47cec808b30") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 11:11:08.557651 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:08.557612    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a89cad11-0010-4ab5-bc82-04a3cd7a3e7b-original-pull-secret\") pod \"global-pull-secret-syncer-px74n\" (UID: \"a89cad11-0010-4ab5-bc82-04a3cd7a3e7b\") " pod="kube-system/global-pull-secret-syncer-px74n"
Jul 07 11:11:08.557834 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:08.557795    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 11:11:08.557893 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:08.557878    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a89cad11-0010-4ab5-bc82-04a3cd7a3e7b-original-pull-secret podName:a89cad11-0010-4ab5-bc82-04a3cd7a3e7b nodeName:}" failed. No retries permitted until 2026-07-07 11:11:24.557857294 +0000 UTC m=+34.267753681 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a89cad11-0010-4ab5-bc82-04a3cd7a3e7b-original-pull-secret") pod "global-pull-secret-syncer-px74n" (UID: "a89cad11-0010-4ab5-bc82-04a3cd7a3e7b") : object "kube-system"/"original-pull-secret" not registered
Jul 07 11:11:08.657992 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:08.657950    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kbzb\" (UniqueName: \"kubernetes.io/projected/f971d15b-e001-45d9-ba6f-e0b60efc55cc-kube-api-access-4kbzb\") pod \"network-check-target-n5mhp\" (UID: \"f971d15b-e001-45d9-ba6f-e0b60efc55cc\") " pod="openshift-network-diagnostics/network-check-target-n5mhp"
Jul 07 11:11:08.658140 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:08.658123    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 11:11:08.658201 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:08.658146    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 11:11:08.658201 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:08.658156    2570 projected.go:196] Error preparing data for projected volume kube-api-access-4kbzb for pod openshift-network-diagnostics/network-check-target-n5mhp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 11:11:08.658274 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:08.658206    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f971d15b-e001-45d9-ba6f-e0b60efc55cc-kube-api-access-4kbzb podName:f971d15b-e001-45d9-ba6f-e0b60efc55cc nodeName:}" failed. No retries permitted until 2026-07-07 11:11:24.65819294 +0000 UTC m=+34.368089305 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-4kbzb" (UniqueName: "kubernetes.io/projected/f971d15b-e001-45d9-ba6f-e0b60efc55cc-kube-api-access-4kbzb") pod "network-check-target-n5mhp" (UID: "f971d15b-e001-45d9-ba6f-e0b60efc55cc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 11:11:08.815024 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:08.814991    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m7hkl"
Jul 07 11:11:08.815146 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:08.815003    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-px74n"
Jul 07 11:11:08.815146 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:08.815102    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m7hkl" podUID="38a7b9ab-6691-41f8-a3b7-e47cec808b30"
Jul 07 11:11:08.815358 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:08.815177    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-px74n" podUID="a89cad11-0010-4ab5-bc82-04a3cd7a3e7b"
Jul 07 11:11:09.815480 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:09.815446    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5mhp"
Jul 07 11:11:09.815932 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:09.815550    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n5mhp" podUID="f971d15b-e001-45d9-ba6f-e0b60efc55cc"
Jul 07 11:11:10.816419 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:10.816378    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m7hkl"
Jul 07 11:11:10.816827 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:10.816504    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m7hkl" podUID="38a7b9ab-6691-41f8-a3b7-e47cec808b30"
Jul 07 11:11:10.816827 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:10.816594    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-px74n"
Jul 07 11:11:10.816827 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:10.816694    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-px74n" podUID="a89cad11-0010-4ab5-bc82-04a3cd7a3e7b"
Jul 07 11:11:11.814582 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:11.814546    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5mhp"
Jul 07 11:11:11.814782 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:11.814684    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n5mhp" podUID="f971d15b-e001-45d9-ba6f-e0b60efc55cc"
Jul 07 11:11:12.815491 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:12.815455    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m7hkl"
Jul 07 11:11:12.815491 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:12.815492    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-px74n"
Jul 07 11:11:12.816018 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:12.815598    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m7hkl" podUID="38a7b9ab-6691-41f8-a3b7-e47cec808b30"
Jul 07 11:11:12.816018 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:12.815692    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-px74n" podUID="a89cad11-0010-4ab5-bc82-04a3cd7a3e7b"
Jul 07 11:11:13.815529 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:13.815179    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5mhp"
Jul 07 11:11:13.816134 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:13.815572    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n5mhp" podUID="f971d15b-e001-45d9-ba6f-e0b60efc55cc"
Jul 07 11:11:14.008071 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:14.007985    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rqrgc" event={"ID":"4ea618ab-a367-4d71-b0bd-e82e15e1f080","Type":"ContainerStarted","Data":"504b337be3d075e95aab74ad47dc0e2403adf5ca179eadb82f52bf3a6a23de78"}
Jul 07 11:11:14.009928 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:14.009890    2570 generic.go:356] "Generic (PLEG): container finished" podID="09a382c5-57c1-4372-92e4-d24a28620c29" containerID="1f9f9623db830c438dbab76935ee31bb8e44957da6003c2297972c00f2b9fb4d" exitCode=0
Jul 07 11:11:14.010070 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:14.009978    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gm84h" event={"ID":"09a382c5-57c1-4372-92e4-d24a28620c29","Type":"ContainerDied","Data":"1f9f9623db830c438dbab76935ee31bb8e44957da6003c2297972c00f2b9fb4d"}
Jul 07 11:11:14.014099 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:14.014073    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9" event={"ID":"ce823451-93f9-4fe7-8ebc-f78efdd21ca2","Type":"ContainerStarted","Data":"74d5414e492825faa90497f46a7b82ee018011c037ac075587cbe614454a9030"}
Jul 07 11:11:14.014205 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:14.014146    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9" event={"ID":"ce823451-93f9-4fe7-8ebc-f78efdd21ca2","Type":"ContainerStarted","Data":"f8502c9c7e1ebfe029574a087d4d5d56db976ed808c21b588c220bc762ba26f8"}
Jul 07 11:11:14.014205 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:14.014194    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9" event={"ID":"ce823451-93f9-4fe7-8ebc-f78efdd21ca2","Type":"ContainerStarted","Data":"023efbaab816df50b91d04d74b88ab4781812bbd944ee2137e8e87cc0bacfae8"}
Jul 07 11:11:14.014321 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:14.014226    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9" event={"ID":"ce823451-93f9-4fe7-8ebc-f78efdd21ca2","Type":"ContainerStarted","Data":"dd10836286c6834a08d24daaa09bd4ab82e90dbd38f14a388578013e1422d265"}
Jul 07 11:11:14.014321 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:14.014262    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9" event={"ID":"ce823451-93f9-4fe7-8ebc-f78efdd21ca2","Type":"ContainerStarted","Data":"d18d09235bf111d0be531c2da3bd4a003f4ab1df97d0221cea1acedcd758f636"}
Jul 07 11:11:14.014321 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:14.014295    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9" event={"ID":"ce823451-93f9-4fe7-8ebc-f78efdd21ca2","Type":"ContainerStarted","Data":"1b0835a79bdd0ab06f907eaafa301f32e405462acb7ab6b2ab5ec2e2a6432c23"}
Jul 07 11:11:14.015867 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:14.015838    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97g4h" event={"ID":"9386a349-2fad-4a2c-88e0-45c997cf3e6b","Type":"ContainerStarted","Data":"b742b3412135226ca25b03f7aa6100d164bd96a02d1a135037a48bd888192982"}
Jul 07 11:11:14.018064 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:14.018039    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bg78n" event={"ID":"7aac7900-132e-4310-90ba-d33718fd0daa","Type":"ContainerStarted","Data":"507813580d8b81b6974756bd841f9cb63da92afda3353d14bfb758667bb1c23e"}
Jul 07 11:11:14.021354 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:14.020981    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gb274" event={"ID":"58b3f62f-e190-4a7c-9a48-c7329e02c146","Type":"ContainerStarted","Data":"af469fcb9de696c6c786c591de6e00ec34d1bfff8a9e001ee0ef1b1e790b4d1d"}
Jul 07 11:11:14.024055 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:14.024033    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dxztq" event={"ID":"c20b78c4-cef9-4f47-ac6e-0ac4303eb857","Type":"ContainerStarted","Data":"397d4a5d6af1344d84f17a9199edf78956af5f225c0370f9a8e8af1bf8c65509"}
Jul 07 11:11:14.025697 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:14.025660    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-rqrgc" podStartSLOduration=3.787358425 podStartE2EDuration="23.025644445s" totalImagesPullingTime="19.23828602s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:10:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:11:14.025192784 +0000 UTC m=+23.735089171" watchObservedRunningTime="2026-07-07 11:11:14.025644445 +0000 UTC m=+23.735540832"
Jul 07 11:11:14.026115 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:14.026091    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-x8c7r" event={"ID":"2827544c-22cd-4454-86a0-fc57d09ea878","Type":"ContainerStarted","Data":"1310aa0df85cfa6a697583da8c9ac63e3222537b9aa3a2cc789ce58017166198"}
Jul 07 11:11:14.046161 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:14.046104    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-gb274" podStartSLOduration=3.789872546 podStartE2EDuration="23.046081849s" totalImagesPullingTime="19.256209303s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:10:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:11:14.045734058 +0000 UTC m=+23.755630447" watchObservedRunningTime="2026-07-07 11:11:14.046081849 +0000 UTC m=+23.755978237"
Jul 07 11:11:14.102627 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:14.102567    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-dxztq" podStartSLOduration=3.865290223 podStartE2EDuration="23.102535134s" totalImagesPullingTime="19.237244911s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:10:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:11:14.102447228 +0000 UTC m=+23.812343614" watchObservedRunningTime="2026-07-07 11:11:14.102535134 +0000 UTC m=+23.812431525"
Jul 07 11:11:14.132349 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:14.132286    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-bg78n" podStartSLOduration=3.847847942 podStartE2EDuration="23.132273739s" totalImagesPullingTime="19.284425797s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:10:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:11:14.131908889 +0000 UTC m=+23.841805288" watchObservedRunningTime="2026-07-07 11:11:14.132273739 +0000 UTC m=+23.842170125"
Jul 07 11:11:14.149639 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:14.149587    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-x8c7r" podStartSLOduration=3.875888108 podStartE2EDuration="23.149571264s" totalImagesPullingTime="19.273683156s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:10:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:11:14.149441325 +0000 UTC m=+23.859337712" watchObservedRunningTime="2026-07-07 11:11:14.149571264 +0000 UTC m=+23.859467650"
Jul 07 11:11:14.814534 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:14.814507    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m7hkl"
Jul 07 11:11:14.814689 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:14.814510    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-px74n"
Jul 07 11:11:14.814689 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:14.814655    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m7hkl" podUID="38a7b9ab-6691-41f8-a3b7-e47cec808b30"
Jul 07 11:11:14.814773 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:14.814708    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-px74n" podUID="a89cad11-0010-4ab5-bc82-04a3cd7a3e7b"
Jul 07 11:11:14.815321 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:14.815300    2570 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 07 11:11:15.030173 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:15.030135    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97g4h" event={"ID":"9386a349-2fad-4a2c-88e0-45c997cf3e6b","Type":"ContainerStarted","Data":"41eb8a548fb9422acbd5b527afd9d856e585346ad358426497e6d27338296c0d"}
Jul 07 11:11:15.034364 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:15.031496    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-x6xq9" event={"ID":"be1eabc6-86f2-4d4f-b598-186f91dcaf4f","Type":"ContainerStarted","Data":"61ed77ea4605b84680b9a6fe29355177b2c9873ab3647dcdc673c7c1e6138d6d"}
Jul 07 11:11:15.058025 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:15.057958    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-x6xq9" podStartSLOduration=4.784961846 podStartE2EDuration="24.057939811s" totalImagesPullingTime="19.272977965s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:10:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:11:15.057732684 +0000 UTC m=+24.767629072" watchObservedRunningTime="2026-07-07 11:11:15.057939811 +0000 UTC m=+24.767836198"
Jul 07 11:11:15.777271 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:15.777163    2570 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T11:11:14.815314462Z","UUID":"100576a4-1098-46b5-bb9f-b8dd17adca97","Handler":null,"Name":"","Endpoint":""}
Jul 07 11:11:15.778805 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:15.778786    2570 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 07 11:11:15.778931 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:15.778818    2570 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 07 11:11:15.814730 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:15.814696    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5mhp"
Jul 07 11:11:15.814939 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:15.814806    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n5mhp" podUID="f971d15b-e001-45d9-ba6f-e0b60efc55cc"
Jul 07 11:11:16.815373 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:16.815103    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-px74n"
Jul 07 11:11:16.815843 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:16.815109    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m7hkl"
Jul 07 11:11:16.815843 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:16.815487    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-px74n" podUID="a89cad11-0010-4ab5-bc82-04a3cd7a3e7b"
Jul 07 11:11:16.815843 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:16.815625    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m7hkl" podUID="38a7b9ab-6691-41f8-a3b7-e47cec808b30"
Jul 07 11:11:17.040191 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:17.040149    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9" event={"ID":"ce823451-93f9-4fe7-8ebc-f78efdd21ca2","Type":"ContainerStarted","Data":"a0acea3f08c37e0105f22a9618a620b68b5f5c42b20d699555ca945a63e38215"}
Jul 07 11:11:17.042439 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:17.042404    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97g4h" event={"ID":"9386a349-2fad-4a2c-88e0-45c997cf3e6b","Type":"ContainerStarted","Data":"4c4e6edf02dd7bc9a9698be8461bbe04682719c2912068617710560fc4ac2e18"}
Jul 07 11:11:17.072888 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:17.072790    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-97g4h" podStartSLOduration=4.2686161590000005 podStartE2EDuration="26.072773911s" totalImagesPullingTime="21.804157752s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:10:51 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:11:17.072463574 +0000 UTC m=+26.782359963" watchObservedRunningTime="2026-07-07 11:11:17.072773911 +0000 UTC m=+26.782670297"
Jul 07 11:11:17.636454 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:17.636425    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-dxztq"
Jul 07 11:11:17.637109 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:17.637085    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-dxztq"
Jul 07 11:11:17.815048 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:17.815017    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5mhp"
Jul 07 11:11:17.815208 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:17.815123    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n5mhp" podUID="f971d15b-e001-45d9-ba6f-e0b60efc55cc"
Jul 07 11:11:18.047362 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:18.046162    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-dxztq"
Jul 07 11:11:18.047362 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:18.046598    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-dxztq"
Jul 07 11:11:18.815219 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:18.814991    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-px74n"
Jul 07 11:11:18.815344 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:18.815001    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m7hkl"
Jul 07 11:11:18.815404 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:18.815354    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-px74n" podUID="a89cad11-0010-4ab5-bc82-04a3cd7a3e7b"
Jul 07 11:11:18.815484 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:18.815397    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m7hkl" podUID="38a7b9ab-6691-41f8-a3b7-e47cec808b30"
Jul 07 11:11:19.049173 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:19.049136    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gm84h" event={"ID":"09a382c5-57c1-4372-92e4-d24a28620c29","Type":"ContainerStarted","Data":"49091bb6e39facc06b68ef45ecd38e597345d35dc73656f3ca8bba72205fe521"}
Jul 07 11:11:19.052699 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:19.052669    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9" event={"ID":"ce823451-93f9-4fe7-8ebc-f78efdd21ca2","Type":"ContainerStarted","Data":"fe11406184d553b41df1b2fbcb5b30004ab6935611d8e1e6842abe95e1d11ceb"}
Jul 07 11:11:19.110355 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:19.110292    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9" podStartSLOduration=8.756055152 podStartE2EDuration="28.110276952s" totalImagesPullingTime="19.3542218s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:10:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:11:19.108688672 +0000 UTC m=+28.818585059" watchObservedRunningTime="2026-07-07 11:11:19.110276952 +0000 UTC m=+28.820173339"
Jul 07 11:11:19.814502 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:19.814461    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5mhp"
Jul 07 11:11:19.814948 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:19.814603    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n5mhp" podUID="f971d15b-e001-45d9-ba6f-e0b60efc55cc"
Jul 07 11:11:20.057058 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:20.057022    2570 generic.go:356] "Generic (PLEG): container finished" podID="09a382c5-57c1-4372-92e4-d24a28620c29" containerID="49091bb6e39facc06b68ef45ecd38e597345d35dc73656f3ca8bba72205fe521" exitCode=0
Jul 07 11:11:20.057569 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:20.057107    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gm84h" event={"ID":"09a382c5-57c1-4372-92e4-d24a28620c29","Type":"ContainerDied","Data":"49091bb6e39facc06b68ef45ecd38e597345d35dc73656f3ca8bba72205fe521"}
Jul 07 11:11:20.058301 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:20.057717    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:11:20.058301 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:20.057790    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:11:20.058301 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:20.057841    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:11:20.074966 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:20.074938    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:11:20.075085 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:20.075041    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:11:20.811583 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:20.811553    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-m7hkl"]
Jul 07 11:11:20.811862 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:20.811663    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m7hkl"
Jul 07 11:11:20.811862 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:20.811768    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m7hkl" podUID="38a7b9ab-6691-41f8-a3b7-e47cec808b30"
Jul 07 11:11:20.815229 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:20.815201    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-px74n"]
Jul 07 11:11:20.816253 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:20.816213    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-px74n"
Jul 07 11:11:20.816377 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:20.816301    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-px74n" podUID="a89cad11-0010-4ab5-bc82-04a3cd7a3e7b"
Jul 07 11:11:20.819576 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:20.819555    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-n5mhp"]
Jul 07 11:11:20.819683 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:20.819666    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5mhp"
Jul 07 11:11:20.819792 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:20.819771    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n5mhp" podUID="f971d15b-e001-45d9-ba6f-e0b60efc55cc"
Jul 07 11:11:22.064208 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:22.064171    2570 generic.go:356] "Generic (PLEG): container finished" podID="09a382c5-57c1-4372-92e4-d24a28620c29" containerID="092f21987cba2fb48ffdeba2576d107ada840c8c348866bbf62e7d1e2d330208" exitCode=0
Jul 07 11:11:22.064693 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:22.064263    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gm84h" event={"ID":"09a382c5-57c1-4372-92e4-d24a28620c29","Type":"ContainerDied","Data":"092f21987cba2fb48ffdeba2576d107ada840c8c348866bbf62e7d1e2d330208"}
Jul 07 11:11:22.815124 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:22.814934    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-px74n"
Jul 07 11:11:22.815340 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:22.814992    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5mhp"
Jul 07 11:11:22.815340 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:22.815218    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-px74n" podUID="a89cad11-0010-4ab5-bc82-04a3cd7a3e7b"
Jul 07 11:11:22.815340 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:22.815260    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n5mhp" podUID="f971d15b-e001-45d9-ba6f-e0b60efc55cc"
Jul 07 11:11:22.815340 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:22.815027    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m7hkl"
Jul 07 11:11:22.815581 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:22.815374    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m7hkl" podUID="38a7b9ab-6691-41f8-a3b7-e47cec808b30"
Jul 07 11:11:24.071587 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:24.071547    2570 generic.go:356] "Generic (PLEG): container finished" podID="09a382c5-57c1-4372-92e4-d24a28620c29" containerID="d0060bc35ece2b8ec6a6e7aecc39c8cd71a369034109d731019f92afa2bf33e1" exitCode=0
Jul 07 11:11:24.072203 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:24.071609    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gm84h" event={"ID":"09a382c5-57c1-4372-92e4-d24a28620c29","Type":"ContainerDied","Data":"d0060bc35ece2b8ec6a6e7aecc39c8cd71a369034109d731019f92afa2bf33e1"}
Jul 07 11:11:24.466962 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:24.466883    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/38a7b9ab-6691-41f8-a3b7-e47cec808b30-metrics-certs\") pod \"network-metrics-daemon-m7hkl\" (UID: \"38a7b9ab-6691-41f8-a3b7-e47cec808b30\") " pod="openshift-multus/network-metrics-daemon-m7hkl"
Jul 07 11:11:24.467094 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:24.466991    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 11:11:24.467094 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:24.467041    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38a7b9ab-6691-41f8-a3b7-e47cec808b30-metrics-certs podName:38a7b9ab-6691-41f8-a3b7-e47cec808b30 nodeName:}" failed. No retries permitted until 2026-07-07 11:11:56.467027512 +0000 UTC m=+66.176923876 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/38a7b9ab-6691-41f8-a3b7-e47cec808b30-metrics-certs") pod "network-metrics-daemon-m7hkl" (UID: "38a7b9ab-6691-41f8-a3b7-e47cec808b30") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 11:11:24.567373 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:24.567341    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a89cad11-0010-4ab5-bc82-04a3cd7a3e7b-original-pull-secret\") pod \"global-pull-secret-syncer-px74n\" (UID: \"a89cad11-0010-4ab5-bc82-04a3cd7a3e7b\") " pod="kube-system/global-pull-secret-syncer-px74n"
Jul 07 11:11:24.567504 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:24.567458    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 11:11:24.567576 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:24.567540    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a89cad11-0010-4ab5-bc82-04a3cd7a3e7b-original-pull-secret podName:a89cad11-0010-4ab5-bc82-04a3cd7a3e7b nodeName:}" failed. No retries permitted until 2026-07-07 11:11:56.567518447 +0000 UTC m=+66.277414829 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a89cad11-0010-4ab5-bc82-04a3cd7a3e7b-original-pull-secret") pod "global-pull-secret-syncer-px74n" (UID: "a89cad11-0010-4ab5-bc82-04a3cd7a3e7b") : object "kube-system"/"original-pull-secret" not registered
Jul 07 11:11:24.667880 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:24.667846    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kbzb\" (UniqueName: \"kubernetes.io/projected/f971d15b-e001-45d9-ba6f-e0b60efc55cc-kube-api-access-4kbzb\") pod \"network-check-target-n5mhp\" (UID: \"f971d15b-e001-45d9-ba6f-e0b60efc55cc\") " pod="openshift-network-diagnostics/network-check-target-n5mhp"
Jul 07 11:11:24.668021 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:24.668008    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 11:11:24.668072 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:24.668027    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 11:11:24.668072 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:24.668036    2570 projected.go:196] Error preparing data for projected volume kube-api-access-4kbzb for pod openshift-network-diagnostics/network-check-target-n5mhp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 11:11:24.668154 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:24.668086    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f971d15b-e001-45d9-ba6f-e0b60efc55cc-kube-api-access-4kbzb podName:f971d15b-e001-45d9-ba6f-e0b60efc55cc nodeName:}" failed. No retries permitted until 2026-07-07 11:11:56.668072229 +0000 UTC m=+66.377968594 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-4kbzb" (UniqueName: "kubernetes.io/projected/f971d15b-e001-45d9-ba6f-e0b60efc55cc-kube-api-access-4kbzb") pod "network-check-target-n5mhp" (UID: "f971d15b-e001-45d9-ba6f-e0b60efc55cc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 11:11:24.814555 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:24.814523    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m7hkl"
Jul 07 11:11:24.814745 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:24.814523    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5mhp"
Jul 07 11:11:24.814745 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:24.814672    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-px74n"
Jul 07 11:11:24.814745 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:24.814673    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m7hkl" podUID="38a7b9ab-6691-41f8-a3b7-e47cec808b30"
Jul 07 11:11:24.814906 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:24.814848    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n5mhp" podUID="f971d15b-e001-45d9-ba6f-e0b60efc55cc"
Jul 07 11:11:24.814946 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:24.814904    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-px74n" podUID="a89cad11-0010-4ab5-bc82-04a3cd7a3e7b"
Jul 07 11:11:26.062044 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.061973    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-39.ec2.internal" event="NodeReady"
Jul 07 11:11:26.062572 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.062124    2570 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 07 11:11:26.101091 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.101059    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-9bb4d966c-zfwhd"]
Jul 07 11:11:26.103644 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.103624    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9bb4d966c-zfwhd"
Jul 07 11:11:26.109300 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.109278    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 07 11:11:26.109495 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.109475    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 07 11:11:26.110112 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.109991    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 07 11:11:26.110367 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.110351    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-hg565\""
Jul 07 11:11:26.117225 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.117206    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 07 11:11:26.119316 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.119295    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-9bb4d966c-zfwhd"]
Jul 07 11:11:26.122366 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.122343    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-64n4d"]
Jul 07 11:11:26.125081 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.125062    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-wlbnq"]
Jul 07 11:11:26.125213 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.125198    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-64n4d"
Jul 07 11:11:26.127800 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.127783    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wlbnq"
Jul 07 11:11:26.128755 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.128604    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2gjl2\""
Jul 07 11:11:26.128755 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.128637    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 07 11:11:26.128957 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.128940    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 07 11:11:26.131611 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.131443    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 07 11:11:26.131611 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.131463    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 07 11:11:26.131772 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.131754    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 07 11:11:26.131833 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.131802    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-jjlcq\""
Jul 07 11:11:26.138983 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.138945    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-64n4d"]
Jul 07 11:11:26.142135 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.142114    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wlbnq"]
Jul 07 11:11:26.279254 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.279210    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cc075655-13c0-4026-9ed2-8c4cc2a71dab-ca-trust-extracted\") pod \"image-registry-9bb4d966c-zfwhd\" (UID: \"cc075655-13c0-4026-9ed2-8c4cc2a71dab\") " pod="openshift-image-registry/image-registry-9bb4d966c-zfwhd"
Jul 07 11:11:26.279440 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.279301    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/daf8c8f2-2b41-4800-a056-1f1b1d9714b7-tmp-dir\") pod \"dns-default-64n4d\" (UID: \"daf8c8f2-2b41-4800-a056-1f1b1d9714b7\") " pod="openshift-dns/dns-default-64n4d"
Jul 07 11:11:26.279440 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.279352    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3e1e8d03-ef0d-49d9-a760-64039277242c-cert\") pod \"ingress-canary-wlbnq\" (UID: \"3e1e8d03-ef0d-49d9-a760-64039277242c\") " pod="openshift-ingress-canary/ingress-canary-wlbnq"
Jul 07 11:11:26.279440 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.279385    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cc075655-13c0-4026-9ed2-8c4cc2a71dab-bound-sa-token\") pod \"image-registry-9bb4d966c-zfwhd\" (UID: \"cc075655-13c0-4026-9ed2-8c4cc2a71dab\") " pod="openshift-image-registry/image-registry-9bb4d966c-zfwhd"
Jul 07 11:11:26.279588 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.279435    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/daf8c8f2-2b41-4800-a056-1f1b1d9714b7-metrics-tls\") pod \"dns-default-64n4d\" (UID: \"daf8c8f2-2b41-4800-a056-1f1b1d9714b7\") " pod="openshift-dns/dns-default-64n4d"
Jul 07 11:11:26.279588 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.279498    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cc075655-13c0-4026-9ed2-8c4cc2a71dab-trusted-ca\") pod \"image-registry-9bb4d966c-zfwhd\" (UID: \"cc075655-13c0-4026-9ed2-8c4cc2a71dab\") " pod="openshift-image-registry/image-registry-9bb4d966c-zfwhd"
Jul 07 11:11:26.279588 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.279539    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cc075655-13c0-4026-9ed2-8c4cc2a71dab-registry-certificates\") pod \"image-registry-9bb4d966c-zfwhd\" (UID: \"cc075655-13c0-4026-9ed2-8c4cc2a71dab\") " pod="openshift-image-registry/image-registry-9bb4d966c-zfwhd"
Jul 07 11:11:26.279588 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.279560    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbskc\" (UniqueName: \"kubernetes.io/projected/daf8c8f2-2b41-4800-a056-1f1b1d9714b7-kube-api-access-bbskc\") pod \"dns-default-64n4d\" (UID: \"daf8c8f2-2b41-4800-a056-1f1b1d9714b7\") " pod="openshift-dns/dns-default-64n4d"
Jul 07 11:11:26.279588 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.279584    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cc075655-13c0-4026-9ed2-8c4cc2a71dab-image-registry-private-configuration\") pod \"image-registry-9bb4d966c-zfwhd\" (UID: \"cc075655-13c0-4026-9ed2-8c4cc2a71dab\") " pod="openshift-image-registry/image-registry-9bb4d966c-zfwhd"
Jul 07 11:11:26.279779 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.279608    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc075655-13c0-4026-9ed2-8c4cc2a71dab-registry-tls\") pod \"image-registry-9bb4d966c-zfwhd\" (UID: \"cc075655-13c0-4026-9ed2-8c4cc2a71dab\") " pod="openshift-image-registry/image-registry-9bb4d966c-zfwhd"
Jul 07 11:11:26.279779 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.279631    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cc075655-13c0-4026-9ed2-8c4cc2a71dab-installation-pull-secrets\") pod \"image-registry-9bb4d966c-zfwhd\" (UID: \"cc075655-13c0-4026-9ed2-8c4cc2a71dab\") " pod="openshift-image-registry/image-registry-9bb4d966c-zfwhd"
Jul 07 11:11:26.279779 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.279714    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/daf8c8f2-2b41-4800-a056-1f1b1d9714b7-config-volume\") pod \"dns-default-64n4d\" (UID: \"daf8c8f2-2b41-4800-a056-1f1b1d9714b7\") " pod="openshift-dns/dns-default-64n4d"
Jul 07 11:11:26.279779 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.279753    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8bz7\" (UniqueName: \"kubernetes.io/projected/cc075655-13c0-4026-9ed2-8c4cc2a71dab-kube-api-access-l8bz7\") pod \"image-registry-9bb4d966c-zfwhd\" (UID: \"cc075655-13c0-4026-9ed2-8c4cc2a71dab\") " pod="openshift-image-registry/image-registry-9bb4d966c-zfwhd"
Jul 07 11:11:26.279914 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.279786    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98gdx\" (UniqueName: \"kubernetes.io/projected/3e1e8d03-ef0d-49d9-a760-64039277242c-kube-api-access-98gdx\") pod \"ingress-canary-wlbnq\" (UID: \"3e1e8d03-ef0d-49d9-a760-64039277242c\") " pod="openshift-ingress-canary/ingress-canary-wlbnq"
Jul 07 11:11:26.334142 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.334058    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxztq_c20b78c4-cef9-4f47-ac6e-0ac4303eb857/konnectivity-agent/0.log" limit=1024
Jul 07 11:11:26.380865 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.380832    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cc075655-13c0-4026-9ed2-8c4cc2a71dab-ca-trust-extracted\") pod \"image-registry-9bb4d966c-zfwhd\" (UID: \"cc075655-13c0-4026-9ed2-8c4cc2a71dab\") " pod="openshift-image-registry/image-registry-9bb4d966c-zfwhd"
Jul 07 11:11:26.380865 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.380873    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/daf8c8f2-2b41-4800-a056-1f1b1d9714b7-tmp-dir\") pod \"dns-default-64n4d\" (UID: \"daf8c8f2-2b41-4800-a056-1f1b1d9714b7\") " pod="openshift-dns/dns-default-64n4d"
Jul 07 11:11:26.381074 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.380897    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3e1e8d03-ef0d-49d9-a760-64039277242c-cert\") pod \"ingress-canary-wlbnq\" (UID: \"3e1e8d03-ef0d-49d9-a760-64039277242c\") " pod="openshift-ingress-canary/ingress-canary-wlbnq"
Jul 07 11:11:26.381074 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.380924    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cc075655-13c0-4026-9ed2-8c4cc2a71dab-bound-sa-token\") pod \"image-registry-9bb4d966c-zfwhd\" (UID: \"cc075655-13c0-4026-9ed2-8c4cc2a71dab\") " pod="openshift-image-registry/image-registry-9bb4d966c-zfwhd"
Jul 07 11:11:26.381074 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.380965    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/daf8c8f2-2b41-4800-a056-1f1b1d9714b7-metrics-tls\") pod \"dns-default-64n4d\" (UID: \"daf8c8f2-2b41-4800-a056-1f1b1d9714b7\") " pod="openshift-dns/dns-default-64n4d"
Jul 07 11:11:26.381074 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.380991    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cc075655-13c0-4026-9ed2-8c4cc2a71dab-trusted-ca\") pod \"image-registry-9bb4d966c-zfwhd\" (UID: \"cc075655-13c0-4026-9ed2-8c4cc2a71dab\") " pod="openshift-image-registry/image-registry-9bb4d966c-zfwhd"
Jul 07 11:11:26.381074 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.381015    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cc075655-13c0-4026-9ed2-8c4cc2a71dab-registry-certificates\") pod \"image-registry-9bb4d966c-zfwhd\" (UID: \"cc075655-13c0-4026-9ed2-8c4cc2a71dab\") " pod="openshift-image-registry/image-registry-9bb4d966c-zfwhd"
Jul 07 11:11:26.381074 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.381037    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbskc\" (UniqueName: \"kubernetes.io/projected/daf8c8f2-2b41-4800-a056-1f1b1d9714b7-kube-api-access-bbskc\") pod \"dns-default-64n4d\" (UID: \"daf8c8f2-2b41-4800-a056-1f1b1d9714b7\") " pod="openshift-dns/dns-default-64n4d"
Jul 07 11:11:26.381074 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.381069    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cc075655-13c0-4026-9ed2-8c4cc2a71dab-image-registry-private-configuration\") pod \"image-registry-9bb4d966c-zfwhd\" (UID: \"cc075655-13c0-4026-9ed2-8c4cc2a71dab\") " pod="openshift-image-registry/image-registry-9bb4d966c-zfwhd"
Jul 07 11:11:26.381296 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.381093    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc075655-13c0-4026-9ed2-8c4cc2a71dab-registry-tls\") pod \"image-registry-9bb4d966c-zfwhd\" (UID: \"cc075655-13c0-4026-9ed2-8c4cc2a71dab\") " pod="openshift-image-registry/image-registry-9bb4d966c-zfwhd"
Jul 07 11:11:26.381296 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.381118    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cc075655-13c0-4026-9ed2-8c4cc2a71dab-installation-pull-secrets\") pod \"image-registry-9bb4d966c-zfwhd\" (UID: \"cc075655-13c0-4026-9ed2-8c4cc2a71dab\") " pod="openshift-image-registry/image-registry-9bb4d966c-zfwhd"
Jul 07 11:11:26.381296 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.381160    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/daf8c8f2-2b41-4800-a056-1f1b1d9714b7-config-volume\") pod \"dns-default-64n4d\" (UID: \"daf8c8f2-2b41-4800-a056-1f1b1d9714b7\") " pod="openshift-dns/dns-default-64n4d"
Jul 07 11:11:26.381296 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.381189    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8bz7\" (UniqueName: \"kubernetes.io/projected/cc075655-13c0-4026-9ed2-8c4cc2a71dab-kube-api-access-l8bz7\") pod \"image-registry-9bb4d966c-zfwhd\" (UID: \"cc075655-13c0-4026-9ed2-8c4cc2a71dab\") " pod="openshift-image-registry/image-registry-9bb4d966c-zfwhd"
Jul 07 11:11:26.381296 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.381214    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98gdx\" (UniqueName: \"kubernetes.io/projected/3e1e8d03-ef0d-49d9-a760-64039277242c-kube-api-access-98gdx\") pod \"ingress-canary-wlbnq\" (UID: \"3e1e8d03-ef0d-49d9-a760-64039277242c\") " pod="openshift-ingress-canary/ingress-canary-wlbnq"
Jul 07 11:11:26.381296 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.381241    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/daf8c8f2-2b41-4800-a056-1f1b1d9714b7-tmp-dir\") pod \"dns-default-64n4d\" (UID: \"daf8c8f2-2b41-4800-a056-1f1b1d9714b7\") " pod="openshift-dns/dns-default-64n4d"
Jul 07 11:11:26.381528 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.381284    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cc075655-13c0-4026-9ed2-8c4cc2a71dab-ca-trust-extracted\") pod \"image-registry-9bb4d966c-zfwhd\" (UID: \"cc075655-13c0-4026-9ed2-8c4cc2a71dab\") " pod="openshift-image-registry/image-registry-9bb4d966c-zfwhd"
Jul 07 11:11:26.381528 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:26.381431    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 11:11:26.381528 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:26.381444    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9bb4d966c-zfwhd: secret "image-registry-tls" not found
Jul 07 11:11:26.381528 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:26.381499    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cc075655-13c0-4026-9ed2-8c4cc2a71dab-registry-tls podName:cc075655-13c0-4026-9ed2-8c4cc2a71dab nodeName:}" failed. No retries permitted until 2026-07-07 11:11:26.881474398 +0000 UTC m=+36.591370770 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cc075655-13c0-4026-9ed2-8c4cc2a71dab-registry-tls") pod "image-registry-9bb4d966c-zfwhd" (UID: "cc075655-13c0-4026-9ed2-8c4cc2a71dab") : secret "image-registry-tls" not found
Jul 07 11:11:26.381736 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:26.381566    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 11:11:26.381736 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:26.381619    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/daf8c8f2-2b41-4800-a056-1f1b1d9714b7-metrics-tls podName:daf8c8f2-2b41-4800-a056-1f1b1d9714b7 nodeName:}" failed. No retries permitted until 2026-07-07 11:11:26.881601576 +0000 UTC m=+36.591497940 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/daf8c8f2-2b41-4800-a056-1f1b1d9714b7-metrics-tls") pod "dns-default-64n4d" (UID: "daf8c8f2-2b41-4800-a056-1f1b1d9714b7") : secret "dns-default-metrics-tls" not found
Jul 07 11:11:26.381991 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.381967    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cc075655-13c0-4026-9ed2-8c4cc2a71dab-registry-certificates\") pod \"image-registry-9bb4d966c-zfwhd\" (UID: \"cc075655-13c0-4026-9ed2-8c4cc2a71dab\") " pod="openshift-image-registry/image-registry-9bb4d966c-zfwhd"
Jul 07 11:11:26.382264 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.382240    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cc075655-13c0-4026-9ed2-8c4cc2a71dab-trusted-ca\") pod \"image-registry-9bb4d966c-zfwhd\" (UID: \"cc075655-13c0-4026-9ed2-8c4cc2a71dab\") " pod="openshift-image-registry/image-registry-9bb4d966c-zfwhd"
Jul 07 11:11:26.382311 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.382258    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/daf8c8f2-2b41-4800-a056-1f1b1d9714b7-config-volume\") pod \"dns-default-64n4d\" (UID: \"daf8c8f2-2b41-4800-a056-1f1b1d9714b7\") " pod="openshift-dns/dns-default-64n4d"
Jul 07 11:11:26.385901 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.385877    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cc075655-13c0-4026-9ed2-8c4cc2a71dab-image-registry-private-configuration\") pod \"image-registry-9bb4d966c-zfwhd\" (UID: \"cc075655-13c0-4026-9ed2-8c4cc2a71dab\") " pod="openshift-image-registry/image-registry-9bb4d966c-zfwhd"
Jul 07 11:11:26.385901 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.385894    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3e1e8d03-ef0d-49d9-a760-64039277242c-cert\") pod \"ingress-canary-wlbnq\" (UID: \"3e1e8d03-ef0d-49d9-a760-64039277242c\") " pod="openshift-ingress-canary/ingress-canary-wlbnq"
Jul 07 11:11:26.386038 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.385899    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cc075655-13c0-4026-9ed2-8c4cc2a71dab-installation-pull-secrets\") pod \"image-registry-9bb4d966c-zfwhd\" (UID: \"cc075655-13c0-4026-9ed2-8c4cc2a71dab\") " pod="openshift-image-registry/image-registry-9bb4d966c-zfwhd"
Jul 07 11:11:26.392728 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.392525    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8bz7\" (UniqueName: \"kubernetes.io/projected/cc075655-13c0-4026-9ed2-8c4cc2a71dab-kube-api-access-l8bz7\") pod \"image-registry-9bb4d966c-zfwhd\" (UID: \"cc075655-13c0-4026-9ed2-8c4cc2a71dab\") " pod="openshift-image-registry/image-registry-9bb4d966c-zfwhd"
Jul 07 11:11:26.392881 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.392859    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cc075655-13c0-4026-9ed2-8c4cc2a71dab-bound-sa-token\") pod \"image-registry-9bb4d966c-zfwhd\" (UID: \"cc075655-13c0-4026-9ed2-8c4cc2a71dab\") " pod="openshift-image-registry/image-registry-9bb4d966c-zfwhd"
Jul 07 11:11:26.393689 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.393658    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-98gdx\" (UniqueName: \"kubernetes.io/projected/3e1e8d03-ef0d-49d9-a760-64039277242c-kube-api-access-98gdx\") pod \"ingress-canary-wlbnq\" (UID: \"3e1e8d03-ef0d-49d9-a760-64039277242c\") " pod="openshift-ingress-canary/ingress-canary-wlbnq"
Jul 07 11:11:26.394585 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.394567    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbskc\" (UniqueName: \"kubernetes.io/projected/daf8c8f2-2b41-4800-a056-1f1b1d9714b7-kube-api-access-bbskc\") pod \"dns-default-64n4d\" (UID: \"daf8c8f2-2b41-4800-a056-1f1b1d9714b7\") " pod="openshift-dns/dns-default-64n4d"
Jul 07 11:11:26.443601 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.443567    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wlbnq"
Jul 07 11:11:26.462825 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.462792    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxztq_c20b78c4-cef9-4f47-ac6e-0ac4303eb857/konnectivity-agent/0.log" limit=1024
Jul 07 11:11:26.602579 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.602483    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wlbnq"]
Jul 07 11:11:26.606581 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:11:26.606555    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e1e8d03_ef0d_49d9_a760_64039277242c.slice/crio-64653630d0ad8039be5300967c0c952b8024debeb19ed5361a9d59a5ab4cd817 WatchSource:0}: Error finding container 64653630d0ad8039be5300967c0c952b8024debeb19ed5361a9d59a5ab4cd817: Status 404 returned error can't find the container with id 64653630d0ad8039be5300967c0c952b8024debeb19ed5361a9d59a5ab4cd817
Jul 07 11:11:26.814630 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.814589    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5mhp"
Jul 07 11:11:26.814816 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.814589    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m7hkl"
Jul 07 11:11:26.815093 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.814589    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-px74n"
Jul 07 11:11:26.818343 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.818314    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 07 11:11:26.818625 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.818605    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7b27w\""
Jul 07 11:11:26.818832 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.818799    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 07 11:11:26.818947 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.818857    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cn2cx\""
Jul 07 11:11:26.818947 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.818939    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 07 11:11:26.819080 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.818941    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 07 11:11:26.886349 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.886249    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/daf8c8f2-2b41-4800-a056-1f1b1d9714b7-metrics-tls\") pod \"dns-default-64n4d\" (UID: \"daf8c8f2-2b41-4800-a056-1f1b1d9714b7\") " pod="openshift-dns/dns-default-64n4d"
Jul 07 11:11:26.886349 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:26.886307    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc075655-13c0-4026-9ed2-8c4cc2a71dab-registry-tls\") pod \"image-registry-9bb4d966c-zfwhd\" (UID: \"cc075655-13c0-4026-9ed2-8c4cc2a71dab\") " pod="openshift-image-registry/image-registry-9bb4d966c-zfwhd"
Jul 07 11:11:26.886566 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:26.886460    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 11:11:26.886566 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:26.886477    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9bb4d966c-zfwhd: secret "image-registry-tls" not found
Jul 07 11:11:26.886566 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:26.886535    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cc075655-13c0-4026-9ed2-8c4cc2a71dab-registry-tls podName:cc075655-13c0-4026-9ed2-8c4cc2a71dab nodeName:}" failed. No retries permitted until 2026-07-07 11:11:27.886519187 +0000 UTC m=+37.596415576 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cc075655-13c0-4026-9ed2-8c4cc2a71dab-registry-tls") pod "image-registry-9bb4d966c-zfwhd" (UID: "cc075655-13c0-4026-9ed2-8c4cc2a71dab") : secret "image-registry-tls" not found
Jul 07 11:11:26.886923 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:26.886908    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 11:11:26.886982 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:26.886965    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/daf8c8f2-2b41-4800-a056-1f1b1d9714b7-metrics-tls podName:daf8c8f2-2b41-4800-a056-1f1b1d9714b7 nodeName:}" failed. No retries permitted until 2026-07-07 11:11:27.886946943 +0000 UTC m=+37.596843313 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/daf8c8f2-2b41-4800-a056-1f1b1d9714b7-metrics-tls") pod "dns-default-64n4d" (UID: "daf8c8f2-2b41-4800-a056-1f1b1d9714b7") : secret "dns-default-metrics-tls" not found
Jul 07 11:11:27.081347 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:27.081294    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wlbnq" event={"ID":"3e1e8d03-ef0d-49d9-a760-64039277242c","Type":"ContainerStarted","Data":"64653630d0ad8039be5300967c0c952b8024debeb19ed5361a9d59a5ab4cd817"}
Jul 07 11:11:27.574467 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:27.574432    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxztq_c20b78c4-cef9-4f47-ac6e-0ac4303eb857/konnectivity-agent/0.log" limit=1024
Jul 07 11:11:27.674988 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:27.674937    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxztq_c20b78c4-cef9-4f47-ac6e-0ac4303eb857/konnectivity-agent/0.log" limit=1024
Jul 07 11:11:27.895884 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:27.895798    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/daf8c8f2-2b41-4800-a056-1f1b1d9714b7-metrics-tls\") pod \"dns-default-64n4d\" (UID: \"daf8c8f2-2b41-4800-a056-1f1b1d9714b7\") " pod="openshift-dns/dns-default-64n4d"
Jul 07 11:11:27.895884 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:27.895846    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc075655-13c0-4026-9ed2-8c4cc2a71dab-registry-tls\") pod \"image-registry-9bb4d966c-zfwhd\" (UID: \"cc075655-13c0-4026-9ed2-8c4cc2a71dab\") " pod="openshift-image-registry/image-registry-9bb4d966c-zfwhd"
Jul 07 11:11:27.896107 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:27.895957    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 11:11:27.896107 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:27.895970    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9bb4d966c-zfwhd: secret "image-registry-tls" not found
Jul 07 11:11:27.896107 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:27.895998    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 11:11:27.896107 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:27.896019    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cc075655-13c0-4026-9ed2-8c4cc2a71dab-registry-tls podName:cc075655-13c0-4026-9ed2-8c4cc2a71dab nodeName:}" failed. No retries permitted until 2026-07-07 11:11:29.896005319 +0000 UTC m=+39.605901685 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cc075655-13c0-4026-9ed2-8c4cc2a71dab-registry-tls") pod "image-registry-9bb4d966c-zfwhd" (UID: "cc075655-13c0-4026-9ed2-8c4cc2a71dab") : secret "image-registry-tls" not found
Jul 07 11:11:27.896107 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:27.896062    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/daf8c8f2-2b41-4800-a056-1f1b1d9714b7-metrics-tls podName:daf8c8f2-2b41-4800-a056-1f1b1d9714b7 nodeName:}" failed. No retries permitted until 2026-07-07 11:11:29.896042735 +0000 UTC m=+39.605939115 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/daf8c8f2-2b41-4800-a056-1f1b1d9714b7-metrics-tls") pod "dns-default-64n4d" (UID: "daf8c8f2-2b41-4800-a056-1f1b1d9714b7") : secret "dns-default-metrics-tls" not found
Jul 07 11:11:28.383487 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.383445    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-wffz6"]
Jul 07 11:11:28.387071 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.387054    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-wffz6"
Jul 07 11:11:28.390202 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.390183    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 07 11:11:28.390762 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.390748    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 07 11:11:28.390823 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.390773    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 07 11:11:28.391749 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.391723    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 07 11:11:28.391844 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.391770    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-mjhsz\""
Jul 07 11:11:28.398825 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.398102    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 07 11:11:28.401349 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.399810    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-wffz6"]
Jul 07 11:11:28.482075 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.482047    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6h7pw"]
Jul 07 11:11:28.486464 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.486436    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6h7pw"
Jul 07 11:11:28.488783 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.488759    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-4v7v8"]
Jul 07 11:11:28.489737 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.489716    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 07 11:11:28.490471 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.490053    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 07 11:11:28.490471 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.490393    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-5h2lm\""
Jul 07 11:11:28.490634 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.490620    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 07 11:11:28.490689 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.490647    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 07 11:11:28.491959 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.491939    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-95lgm"]
Jul 07 11:11:28.492100 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.492072    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-4v7v8"
Jul 07 11:11:28.495888 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.495867    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-5dc4cf6785-bflm8"]
Jul 07 11:11:28.496301 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.496284    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-95lgm"
Jul 07 11:11:28.499158 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.498761    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-tt42z"]
Jul 07 11:11:28.499158 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.498825    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 07 11:11:28.499158 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.499092    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 07 11:11:28.499380 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.499303    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 07 11:11:28.500857 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.499603    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 07 11:11:28.500857 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.499665    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5dc4cf6785-bflm8"
Jul 07 11:11:28.500857 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.500535    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6387d3da-1290-437c-8fdc-2422d49dac72-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-wffz6\" (UID: \"6387d3da-1290-437c-8fdc-2422d49dac72\") " pod="openshift-insights/insights-operator-5c847dcfc4-wffz6"
Jul 07 11:11:28.500857 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.500588    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/6387d3da-1290-437c-8fdc-2422d49dac72-snapshots\") pod \"insights-operator-5c847dcfc4-wffz6\" (UID: \"6387d3da-1290-437c-8fdc-2422d49dac72\") " pod="openshift-insights/insights-operator-5c847dcfc4-wffz6"
Jul 07 11:11:28.500857 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.500647    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6387d3da-1290-437c-8fdc-2422d49dac72-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-wffz6\" (UID: \"6387d3da-1290-437c-8fdc-2422d49dac72\") " pod="openshift-insights/insights-operator-5c847dcfc4-wffz6"
Jul 07 11:11:28.500857 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.500714    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6387d3da-1290-437c-8fdc-2422d49dac72-serving-cert\") pod \"insights-operator-5c847dcfc4-wffz6\" (UID: \"6387d3da-1290-437c-8fdc-2422d49dac72\") " pod="openshift-insights/insights-operator-5c847dcfc4-wffz6"
Jul 07 11:11:28.500857 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.500738    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2h98\" (UniqueName: \"kubernetes.io/projected/6387d3da-1290-437c-8fdc-2422d49dac72-kube-api-access-g2h98\") pod \"insights-operator-5c847dcfc4-wffz6\" (UID: \"6387d3da-1290-437c-8fdc-2422d49dac72\") " pod="openshift-insights/insights-operator-5c847dcfc4-wffz6"
Jul 07 11:11:28.500857 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.500767    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6387d3da-1290-437c-8fdc-2422d49dac72-tmp\") pod \"insights-operator-5c847dcfc4-wffz6\" (UID: \"6387d3da-1290-437c-8fdc-2422d49dac72\") " pod="openshift-insights/insights-operator-5c847dcfc4-wffz6"
Jul 07 11:11:28.501524 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.501495    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 07 11:11:28.501781 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.501738    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-gzvnv\""
Jul 07 11:11:28.501946 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.501855    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 07 11:11:28.502153 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.502135    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 07 11:11:28.502241 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.502166    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-7bnf6\""
Jul 07 11:11:28.503403 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.502355    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 07 11:11:28.503403 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.503296    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6h7pw"]
Jul 07 11:11:28.503526 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.503423    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-tt42z"
Jul 07 11:11:28.503584 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.503553    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 07 11:11:28.503920 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.503638    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 07 11:11:28.503920 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.503747    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-zdx2d\""
Jul 07 11:11:28.503920 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.503802    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 07 11:11:28.503920 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.503858    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 07 11:11:28.504372 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.504344    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 07 11:11:28.505250 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.505166    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 07 11:11:28.514067 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.512669    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 07 11:11:28.514067 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.513617    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-5tqss\""
Jul 07 11:11:28.514067 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.513991    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 07 11:11:28.515531 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.515473    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-tt42z"]
Jul 07 11:11:28.516243 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.516226    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 07 11:11:28.516564 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.516528    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-4v7v8"]
Jul 07 11:11:28.518892 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.518819    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-95lgm"]
Jul 07 11:11:28.525348 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.525307    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5dc4cf6785-bflm8"]
Jul 07 11:11:28.586538 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.586509    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-9pdtd"]
Jul 07 11:11:28.605526 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.601814    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2h98\" (UniqueName: \"kubernetes.io/projected/6387d3da-1290-437c-8fdc-2422d49dac72-kube-api-access-g2h98\") pod \"insights-operator-5c847dcfc4-wffz6\" (UID: \"6387d3da-1290-437c-8fdc-2422d49dac72\") " pod="openshift-insights/insights-operator-5c847dcfc4-wffz6"
Jul 07 11:11:28.605526 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.602016    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6387d3da-1290-437c-8fdc-2422d49dac72-tmp\") pod \"insights-operator-5c847dcfc4-wffz6\" (UID: \"6387d3da-1290-437c-8fdc-2422d49dac72\") " pod="openshift-insights/insights-operator-5c847dcfc4-wffz6"
Jul 07 11:11:28.605526 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.602069    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c8c17c90-3e07-4530-8411-d59cef9a1aff-config\") pod \"console-operator-68b757fb67-4v7v8\" (UID: \"c8c17c90-3e07-4530-8411-d59cef9a1aff\") " pod="openshift-console-operator/console-operator-68b757fb67-4v7v8"
Jul 07 11:11:28.605526 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.602237    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c8c17c90-3e07-4530-8411-d59cef9a1aff-trusted-ca\") pod \"console-operator-68b757fb67-4v7v8\" (UID: \"c8c17c90-3e07-4530-8411-d59cef9a1aff\") " pod="openshift-console-operator/console-operator-68b757fb67-4v7v8"
Jul 07 11:11:28.605526 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.602272    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8bns\" (UniqueName: \"kubernetes.io/projected/91203aa4-b720-4390-a2c0-5c553ffd79f8-kube-api-access-x8bns\") pod \"router-default-5dc4cf6785-bflm8\" (UID: \"91203aa4-b720-4390-a2c0-5c553ffd79f8\") " pod="openshift-ingress/router-default-5dc4cf6785-bflm8"
Jul 07 11:11:28.605526 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.602498    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c8c17c90-3e07-4530-8411-d59cef9a1aff-serving-cert\") pod \"console-operator-68b757fb67-4v7v8\" (UID: \"c8c17c90-3e07-4530-8411-d59cef9a1aff\") " pod="openshift-console-operator/console-operator-68b757fb67-4v7v8"
Jul 07 11:11:28.605526 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.602541    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7fba6280-ffbe-4a71-98f3-3432b0584697-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-6h7pw\" (UID: \"7fba6280-ffbe-4a71-98f3-3432b0584697\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6h7pw"
Jul 07 11:11:28.605526 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.602715    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7fba6280-ffbe-4a71-98f3-3432b0584697-config\") pod \"service-ca-operator-5d46d9ffd4-6h7pw\" (UID: \"7fba6280-ffbe-4a71-98f3-3432b0584697\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6h7pw"
Jul 07 11:11:28.605526 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.602759    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/6387d3da-1290-437c-8fdc-2422d49dac72-snapshots\") pod \"insights-operator-5c847dcfc4-wffz6\" (UID: \"6387d3da-1290-437c-8fdc-2422d49dac72\") " pod="openshift-insights/insights-operator-5c847dcfc4-wffz6"
Jul 07 11:11:28.605526 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.602952    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6387d3da-1290-437c-8fdc-2422d49dac72-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-wffz6\" (UID: \"6387d3da-1290-437c-8fdc-2422d49dac72\") " pod="openshift-insights/insights-operator-5c847dcfc4-wffz6"
Jul 07 11:11:28.605526 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.602992    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6387d3da-1290-437c-8fdc-2422d49dac72-serving-cert\") pod \"insights-operator-5c847dcfc4-wffz6\" (UID: \"6387d3da-1290-437c-8fdc-2422d49dac72\") " pod="openshift-insights/insights-operator-5c847dcfc4-wffz6"
Jul 07 11:11:28.605526 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.603150    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/192f418c-f6e6-4775-876f-ef3d4f11b265-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-95lgm\" (UID: \"192f418c-f6e6-4775-876f-ef3d4f11b265\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-95lgm"
Jul 07 11:11:28.605526 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.603318    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v42s8\" (UniqueName: \"kubernetes.io/projected/7fba6280-ffbe-4a71-98f3-3432b0584697-kube-api-access-v42s8\") pod \"service-ca-operator-5d46d9ffd4-6h7pw\" (UID: \"7fba6280-ffbe-4a71-98f3-3432b0584697\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6h7pw"
Jul 07 11:11:28.605526 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.603393    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/91203aa4-b720-4390-a2c0-5c553ffd79f8-default-certificate\") pod \"router-default-5dc4cf6785-bflm8\" (UID: \"91203aa4-b720-4390-a2c0-5c553ffd79f8\") " pod="openshift-ingress/router-default-5dc4cf6785-bflm8"
Jul 07 11:11:28.605526 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.603428    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91203aa4-b720-4390-a2c0-5c553ffd79f8-metrics-certs\") pod \"router-default-5dc4cf6785-bflm8\" (UID: \"91203aa4-b720-4390-a2c0-5c553ffd79f8\") " pod="openshift-ingress/router-default-5dc4cf6785-bflm8"
Jul 07 11:11:28.606308 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.603458    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gssk\" (UniqueName: \"kubernetes.io/projected/192f418c-f6e6-4775-876f-ef3d4f11b265-kube-api-access-8gssk\") pod \"kube-storage-version-migrator-operator-846bf668f7-95lgm\" (UID: \"192f418c-f6e6-4775-876f-ef3d4f11b265\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-95lgm"
Jul 07 11:11:28.606308 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.603543    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91203aa4-b720-4390-a2c0-5c553ffd79f8-service-ca-bundle\") pod \"router-default-5dc4cf6785-bflm8\" (UID: \"91203aa4-b720-4390-a2c0-5c553ffd79f8\") " pod="openshift-ingress/router-default-5dc4cf6785-bflm8"
Jul 07 11:11:28.606308 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.603577    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqj9r\" (UniqueName: \"kubernetes.io/projected/c8c17c90-3e07-4530-8411-d59cef9a1aff-kube-api-access-xqj9r\") pod \"console-operator-68b757fb67-4v7v8\" (UID: \"c8c17c90-3e07-4530-8411-d59cef9a1aff\") " pod="openshift-console-operator/console-operator-68b757fb67-4v7v8"
Jul 07 11:11:28.606308 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.603729    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6387d3da-1290-437c-8fdc-2422d49dac72-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-wffz6\" (UID: \"6387d3da-1290-437c-8fdc-2422d49dac72\") " pod="openshift-insights/insights-operator-5c847dcfc4-wffz6"
Jul 07 11:11:28.606308 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.603872    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/91203aa4-b720-4390-a2c0-5c553ffd79f8-stats-auth\") pod \"router-default-5dc4cf6785-bflm8\" (UID: \"91203aa4-b720-4390-a2c0-5c553ffd79f8\") " pod="openshift-ingress/router-default-5dc4cf6785-bflm8"
Jul 07 11:11:28.606308 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.603957    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/192f418c-f6e6-4775-876f-ef3d4f11b265-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-95lgm\" (UID: \"192f418c-f6e6-4775-876f-ef3d4f11b265\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-95lgm"
Jul 07 11:11:28.606308 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.604111    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85tgp\" (UniqueName: \"kubernetes.io/projected/62ea12c1-c37b-4e29-9700-bd826f004e51-kube-api-access-85tgp\") pod \"volume-data-source-validator-5d456fdbcd-tt42z\" (UID: \"62ea12c1-c37b-4e29-9700-bd826f004e51\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-tt42z"
Jul 07 11:11:28.606308 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.604673    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-r6v22"]
Jul 07 11:11:28.607218 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.607188    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6387d3da-1290-437c-8fdc-2422d49dac72-tmp\") pod \"insights-operator-5c847dcfc4-wffz6\" (UID: \"6387d3da-1290-437c-8fdc-2422d49dac72\") " pod="openshift-insights/insights-operator-5c847dcfc4-wffz6"
Jul 07 11:11:28.607830 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.607808    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/6387d3da-1290-437c-8fdc-2422d49dac72-snapshots\") pod \"insights-operator-5c847dcfc4-wffz6\" (UID: \"6387d3da-1290-437c-8fdc-2422d49dac72\") " pod="openshift-insights/insights-operator-5c847dcfc4-wffz6"
Jul 07 11:11:28.609238 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.608352    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-9pdtd"
Jul 07 11:11:28.609238 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.608864    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6387d3da-1290-437c-8fdc-2422d49dac72-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-wffz6\" (UID: \"6387d3da-1290-437c-8fdc-2422d49dac72\") " pod="openshift-insights/insights-operator-5c847dcfc4-wffz6"
Jul 07 11:11:28.610258 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.610113    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6387d3da-1290-437c-8fdc-2422d49dac72-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-wffz6\" (UID: \"6387d3da-1290-437c-8fdc-2422d49dac72\") " pod="openshift-insights/insights-operator-5c847dcfc4-wffz6"
Jul 07 11:11:28.613847 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.613719    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6387d3da-1290-437c-8fdc-2422d49dac72-serving-cert\") pod \"insights-operator-5c847dcfc4-wffz6\" (UID: \"6387d3da-1290-437c-8fdc-2422d49dac72\") " pod="openshift-insights/insights-operator-5c847dcfc4-wffz6"
Jul 07 11:11:28.614110 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.614087    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-chkxh\""
Jul 07 11:11:28.614227 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.614153    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 07 11:11:28.614227 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.614201    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 07 11:11:28.614354 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.614299    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 07 11:11:28.614415 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.614152    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 07 11:11:28.620249 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.620227    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2h98\" (UniqueName: \"kubernetes.io/projected/6387d3da-1290-437c-8fdc-2422d49dac72-kube-api-access-g2h98\") pod \"insights-operator-5c847dcfc4-wffz6\" (UID: \"6387d3da-1290-437c-8fdc-2422d49dac72\") " pod="openshift-insights/insights-operator-5c847dcfc4-wffz6"
Jul 07 11:11:28.627682 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.627660    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-vj4hq"]
Jul 07 11:11:28.627903 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.627803    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-r6v22"
Jul 07 11:11:28.630698 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.630678    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-pr94m\""
Jul 07 11:11:28.651640 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.651619    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-lmnhq"]
Jul 07 11:11:28.651764 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.651736    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-vj4hq"
Jul 07 11:11:28.654439 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.654414    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxztq_c20b78c4-cef9-4f47-ac6e-0ac4303eb857/konnectivity-agent/0.log" limit=1024
Jul 07 11:11:28.654546 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.654489    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 07 11:11:28.654598 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.654546    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-5lm6j\""
Jul 07 11:11:28.654734 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.654721    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 07 11:11:28.676694 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.676674    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-r6v22"]
Jul 07 11:11:28.676826 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.676721    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-9pdtd"]
Jul 07 11:11:28.676826 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.676740    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-vj4hq"]
Jul 07 11:11:28.676826 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.676758    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-lmnhq"]
Jul 07 11:11:28.676826 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.676760    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-lmnhq"
Jul 07 11:11:28.679442 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.679420    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 07 11:11:28.679567 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.679458    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 07 11:11:28.679813 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.679601    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 07 11:11:28.679813 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.679654    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-jr4xl\""
Jul 07 11:11:28.707444 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.707416    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/192f418c-f6e6-4775-876f-ef3d4f11b265-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-95lgm\" (UID: \"192f418c-f6e6-4775-876f-ef3d4f11b265\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-95lgm"
Jul 07 11:11:28.707606 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.707458    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v42s8\" (UniqueName: \"kubernetes.io/projected/7fba6280-ffbe-4a71-98f3-3432b0584697-kube-api-access-v42s8\") pod \"service-ca-operator-5d46d9ffd4-6h7pw\" (UID: \"7fba6280-ffbe-4a71-98f3-3432b0584697\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6h7pw"
Jul 07 11:11:28.707606 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.707485    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/91203aa4-b720-4390-a2c0-5c553ffd79f8-default-certificate\") pod \"router-default-5dc4cf6785-bflm8\" (UID: \"91203aa4-b720-4390-a2c0-5c553ffd79f8\") " pod="openshift-ingress/router-default-5dc4cf6785-bflm8"
Jul 07 11:11:28.707606 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.707508    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91203aa4-b720-4390-a2c0-5c553ffd79f8-metrics-certs\") pod \"router-default-5dc4cf6785-bflm8\" (UID: \"91203aa4-b720-4390-a2c0-5c553ffd79f8\") " pod="openshift-ingress/router-default-5dc4cf6785-bflm8"
Jul 07 11:11:28.707606 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.707533    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gssk\" (UniqueName: \"kubernetes.io/projected/192f418c-f6e6-4775-876f-ef3d4f11b265-kube-api-access-8gssk\") pod \"kube-storage-version-migrator-operator-846bf668f7-95lgm\" (UID: \"192f418c-f6e6-4775-876f-ef3d4f11b265\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-95lgm"
Jul 07 11:11:28.707606 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.707557    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91203aa4-b720-4390-a2c0-5c553ffd79f8-service-ca-bundle\") pod \"router-default-5dc4cf6785-bflm8\" (UID: \"91203aa4-b720-4390-a2c0-5c553ffd79f8\") " pod="openshift-ingress/router-default-5dc4cf6785-bflm8"
Jul 07 11:11:28.707606 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.707597    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqj9r\" (UniqueName: \"kubernetes.io/projected/c8c17c90-3e07-4530-8411-d59cef9a1aff-kube-api-access-xqj9r\") pod \"console-operator-68b757fb67-4v7v8\" (UID: \"c8c17c90-3e07-4530-8411-d59cef9a1aff\") " pod="openshift-console-operator/console-operator-68b757fb67-4v7v8"
Jul 07 11:11:28.707914 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.707639    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/91203aa4-b720-4390-a2c0-5c553ffd79f8-stats-auth\") pod \"router-default-5dc4cf6785-bflm8\" (UID: \"91203aa4-b720-4390-a2c0-5c553ffd79f8\") " pod="openshift-ingress/router-default-5dc4cf6785-bflm8"
Jul 07 11:11:28.707914 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.707670    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9j6cq\" (UniqueName: \"kubernetes.io/projected/df6d16a6-2593-4477-ae7a-eca848817f34-kube-api-access-9j6cq\") pod \"cluster-monitoring-operator-64786d9cbb-9pdtd\" (UID: \"df6d16a6-2593-4477-ae7a-eca848817f34\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-9pdtd"
Jul 07 11:11:28.707914 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.707713    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/192f418c-f6e6-4775-876f-ef3d4f11b265-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-95lgm\" (UID: \"192f418c-f6e6-4775-876f-ef3d4f11b265\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-95lgm"
Jul 07 11:11:28.707914 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.707742    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85tgp\" (UniqueName: \"kubernetes.io/projected/62ea12c1-c37b-4e29-9700-bd826f004e51-kube-api-access-85tgp\") pod \"volume-data-source-validator-5d456fdbcd-tt42z\" (UID: \"62ea12c1-c37b-4e29-9700-bd826f004e51\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-tt42z"
Jul 07 11:11:28.707914 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.707791    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c8c17c90-3e07-4530-8411-d59cef9a1aff-config\") pod \"console-operator-68b757fb67-4v7v8\" (UID: \"c8c17c90-3e07-4530-8411-d59cef9a1aff\") " pod="openshift-console-operator/console-operator-68b757fb67-4v7v8"
Jul 07 11:11:28.707914 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.707820    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/df6d16a6-2593-4477-ae7a-eca848817f34-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-9pdtd\" (UID: \"df6d16a6-2593-4477-ae7a-eca848817f34\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-9pdtd"
Jul 07 11:11:28.707914 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.707848    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c8c17c90-3e07-4530-8411-d59cef9a1aff-trusted-ca\") pod \"console-operator-68b757fb67-4v7v8\" (UID: \"c8c17c90-3e07-4530-8411-d59cef9a1aff\") " pod="openshift-console-operator/console-operator-68b757fb67-4v7v8"
Jul 07 11:11:28.707914 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.707875    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8bns\" (UniqueName: \"kubernetes.io/projected/91203aa4-b720-4390-a2c0-5c553ffd79f8-kube-api-access-x8bns\") pod \"router-default-5dc4cf6785-bflm8\" (UID: \"91203aa4-b720-4390-a2c0-5c553ffd79f8\") " pod="openshift-ingress/router-default-5dc4cf6785-bflm8"
Jul 07 11:11:28.707914 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.707912    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c8c17c90-3e07-4530-8411-d59cef9a1aff-serving-cert\") pod \"console-operator-68b757fb67-4v7v8\" (UID: \"c8c17c90-3e07-4530-8411-d59cef9a1aff\") " pod="openshift-console-operator/console-operator-68b757fb67-4v7v8"
Jul 07 11:11:28.708310 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.707935    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7fba6280-ffbe-4a71-98f3-3432b0584697-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-6h7pw\" (UID: \"7fba6280-ffbe-4a71-98f3-3432b0584697\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6h7pw"
Jul 07 11:11:28.708310 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.707968    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/df6d16a6-2593-4477-ae7a-eca848817f34-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-9pdtd\" (UID: \"df6d16a6-2593-4477-ae7a-eca848817f34\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-9pdtd"
Jul 07 11:11:28.708310 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.707995    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7fba6280-ffbe-4a71-98f3-3432b0584697-config\") pod \"service-ca-operator-5d46d9ffd4-6h7pw\" (UID: \"7fba6280-ffbe-4a71-98f3-3432b0584697\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6h7pw"
Jul 07 11:11:28.708310 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:28.708165    2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 07 11:11:28.708310 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:28.708217    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/91203aa4-b720-4390-a2c0-5c553ffd79f8-service-ca-bundle podName:91203aa4-b720-4390-a2c0-5c553ffd79f8 nodeName:}" failed. No retries permitted until 2026-07-07 11:11:29.208199317 +0000 UTC m=+38.918095682 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/91203aa4-b720-4390-a2c0-5c553ffd79f8-service-ca-bundle") pod "router-default-5dc4cf6785-bflm8" (UID: "91203aa4-b720-4390-a2c0-5c553ffd79f8") : configmap references non-existent config key: service-ca.crt
Jul 07 11:11:28.708310 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:28.708242    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91203aa4-b720-4390-a2c0-5c553ffd79f8-metrics-certs podName:91203aa4-b720-4390-a2c0-5c553ffd79f8 nodeName:}" failed. No retries permitted until 2026-07-07 11:11:29.208231528 +0000 UTC m=+38.918127893 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/91203aa4-b720-4390-a2c0-5c553ffd79f8-metrics-certs") pod "router-default-5dc4cf6785-bflm8" (UID: "91203aa4-b720-4390-a2c0-5c553ffd79f8") : secret "router-metrics-certs-default" not found
Jul 07 11:11:28.709070 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.708848    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/192f418c-f6e6-4775-876f-ef3d4f11b265-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-95lgm\" (UID: \"192f418c-f6e6-4775-876f-ef3d4f11b265\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-95lgm"
Jul 07 11:11:28.709596 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.709576    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7fba6280-ffbe-4a71-98f3-3432b0584697-config\") pod \"service-ca-operator-5d46d9ffd4-6h7pw\" (UID: \"7fba6280-ffbe-4a71-98f3-3432b0584697\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6h7pw"
Jul 07 11:11:28.709744 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.709592    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c8c17c90-3e07-4530-8411-d59cef9a1aff-config\") pod \"console-operator-68b757fb67-4v7v8\" (UID: \"c8c17c90-3e07-4530-8411-d59cef9a1aff\") " pod="openshift-console-operator/console-operator-68b757fb67-4v7v8"
Jul 07 11:11:28.709869 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.709795    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c8c17c90-3e07-4530-8411-d59cef9a1aff-trusted-ca\") pod \"console-operator-68b757fb67-4v7v8\" (UID: \"c8c17c90-3e07-4530-8411-d59cef9a1aff\") " pod="openshift-console-operator/console-operator-68b757fb67-4v7v8"
Jul 07 11:11:28.710727 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.710707    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-wffz6"
Jul 07 11:11:28.711424 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.711406    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c8c17c90-3e07-4530-8411-d59cef9a1aff-serving-cert\") pod \"console-operator-68b757fb67-4v7v8\" (UID: \"c8c17c90-3e07-4530-8411-d59cef9a1aff\") " pod="openshift-console-operator/console-operator-68b757fb67-4v7v8"
Jul 07 11:11:28.711621 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.711603    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/91203aa4-b720-4390-a2c0-5c553ffd79f8-default-certificate\") pod \"router-default-5dc4cf6785-bflm8\" (UID: \"91203aa4-b720-4390-a2c0-5c553ffd79f8\") " pod="openshift-ingress/router-default-5dc4cf6785-bflm8"
Jul 07 11:11:28.712054 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.712029    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7fba6280-ffbe-4a71-98f3-3432b0584697-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-6h7pw\" (UID: \"7fba6280-ffbe-4a71-98f3-3432b0584697\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6h7pw"
Jul 07 11:11:28.712402 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.712339    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/192f418c-f6e6-4775-876f-ef3d4f11b265-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-95lgm\" (UID: \"192f418c-f6e6-4775-876f-ef3d4f11b265\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-95lgm"
Jul 07 11:11:28.713070 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.713052    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/91203aa4-b720-4390-a2c0-5c553ffd79f8-stats-auth\") pod \"router-default-5dc4cf6785-bflm8\" (UID: \"91203aa4-b720-4390-a2c0-5c553ffd79f8\") " pod="openshift-ingress/router-default-5dc4cf6785-bflm8"
Jul 07 11:11:28.720910 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.720862    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqj9r\" (UniqueName: \"kubernetes.io/projected/c8c17c90-3e07-4530-8411-d59cef9a1aff-kube-api-access-xqj9r\") pod \"console-operator-68b757fb67-4v7v8\" (UID: \"c8c17c90-3e07-4530-8411-d59cef9a1aff\") " pod="openshift-console-operator/console-operator-68b757fb67-4v7v8"
Jul 07 11:11:28.723371 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.723303    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gssk\" (UniqueName: \"kubernetes.io/projected/192f418c-f6e6-4775-876f-ef3d4f11b265-kube-api-access-8gssk\") pod \"kube-storage-version-migrator-operator-846bf668f7-95lgm\" (UID: \"192f418c-f6e6-4775-876f-ef3d4f11b265\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-95lgm"
Jul 07 11:11:28.723506 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.723434    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxztq_c20b78c4-cef9-4f47-ac6e-0ac4303eb857/konnectivity-agent/0.log" limit=1024
Jul 07 11:11:28.723918 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.723893    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v42s8\" (UniqueName: \"kubernetes.io/projected/7fba6280-ffbe-4a71-98f3-3432b0584697-kube-api-access-v42s8\") pod \"service-ca-operator-5d46d9ffd4-6h7pw\" (UID: \"7fba6280-ffbe-4a71-98f3-3432b0584697\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6h7pw"
Jul 07 11:11:28.724414 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.724393    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-85tgp\" (UniqueName: \"kubernetes.io/projected/62ea12c1-c37b-4e29-9700-bd826f004e51-kube-api-access-85tgp\") pod \"volume-data-source-validator-5d456fdbcd-tt42z\" (UID: \"62ea12c1-c37b-4e29-9700-bd826f004e51\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-tt42z"
Jul 07 11:11:28.725713 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.725587    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8bns\" (UniqueName: \"kubernetes.io/projected/91203aa4-b720-4390-a2c0-5c553ffd79f8-kube-api-access-x8bns\") pod \"router-default-5dc4cf6785-bflm8\" (UID: \"91203aa4-b720-4390-a2c0-5c553ffd79f8\") " pod="openshift-ingress/router-default-5dc4cf6785-bflm8"
Jul 07 11:11:28.801657 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.801631    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6h7pw"
Jul 07 11:11:28.808979 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.808944    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9j6cq\" (UniqueName: \"kubernetes.io/projected/df6d16a6-2593-4477-ae7a-eca848817f34-kube-api-access-9j6cq\") pod \"cluster-monitoring-operator-64786d9cbb-9pdtd\" (UID: \"df6d16a6-2593-4477-ae7a-eca848817f34\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-9pdtd"
Jul 07 11:11:28.809125 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.808985    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f9ec8255-2877-4c6a-b248-b50d64978f7a-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-vj4hq\" (UID: \"f9ec8255-2877-4c6a-b248-b50d64978f7a\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-vj4hq"
Jul 07 11:11:28.809125 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.809013    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8kng\" (UniqueName: \"kubernetes.io/projected/63454a74-dfe7-497f-9f3d-0725bc111250-kube-api-access-v8kng\") pod \"cluster-samples-operator-56fcf86bf-lmnhq\" (UID: \"63454a74-dfe7-497f-9f3d-0725bc111250\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-lmnhq"
Jul 07 11:11:28.809125 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.809077    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/df6d16a6-2593-4477-ae7a-eca848817f34-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-9pdtd\" (UID: \"df6d16a6-2593-4477-ae7a-eca848817f34\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-9pdtd"
Jul 07 11:11:28.809125 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.809119    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f9ec8255-2877-4c6a-b248-b50d64978f7a-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-vj4hq\" (UID: \"f9ec8255-2877-4c6a-b248-b50d64978f7a\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-vj4hq"
Jul 07 11:11:28.809256 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.809140    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpp89\" (UniqueName: \"kubernetes.io/projected/0d017aa6-bcf4-4ff7-9161-e477849c7f10-kube-api-access-dpp89\") pod \"network-check-source-989f8b8dc-r6v22\" (UID: \"0d017aa6-bcf4-4ff7-9161-e477849c7f10\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-r6v22"
Jul 07 11:11:28.809256 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.809163    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/df6d16a6-2593-4477-ae7a-eca848817f34-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-9pdtd\" (UID: \"df6d16a6-2593-4477-ae7a-eca848817f34\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-9pdtd"
Jul 07 11:11:28.809256 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.809185    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/63454a74-dfe7-497f-9f3d-0725bc111250-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-lmnhq\" (UID: \"63454a74-dfe7-497f-9f3d-0725bc111250\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-lmnhq"
Jul 07 11:11:28.809386 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:28.809372    2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 07 11:11:28.809433 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:28.809424    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df6d16a6-2593-4477-ae7a-eca848817f34-cluster-monitoring-operator-tls podName:df6d16a6-2593-4477-ae7a-eca848817f34 nodeName:}" failed. No retries permitted until 2026-07-07 11:11:29.309408263 +0000 UTC m=+39.019304627 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/df6d16a6-2593-4477-ae7a-eca848817f34-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-9pdtd" (UID: "df6d16a6-2593-4477-ae7a-eca848817f34") : secret "cluster-monitoring-operator-tls" not found
Jul 07 11:11:28.809819 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.809797    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/df6d16a6-2593-4477-ae7a-eca848817f34-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-9pdtd\" (UID: \"df6d16a6-2593-4477-ae7a-eca848817f34\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-9pdtd"
Jul 07 11:11:28.820294 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.820191    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9j6cq\" (UniqueName: \"kubernetes.io/projected/df6d16a6-2593-4477-ae7a-eca848817f34-kube-api-access-9j6cq\") pod \"cluster-monitoring-operator-64786d9cbb-9pdtd\" (UID: \"df6d16a6-2593-4477-ae7a-eca848817f34\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-9pdtd"
Jul 07 11:11:28.856806 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.856778    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-4v7v8"
Jul 07 11:11:28.864910 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.864881    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-95lgm"
Jul 07 11:11:28.872205 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.872176    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-wffz6"]
Jul 07 11:11:28.875739 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:11:28.875712    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6387d3da_1290_437c_8fdc_2422d49dac72.slice/crio-a7c9fbcb2cdba937cd1c43d8fd0ee6d626ef6b883535e7f7aec2cbb1863e27de WatchSource:0}: Error finding container a7c9fbcb2cdba937cd1c43d8fd0ee6d626ef6b883535e7f7aec2cbb1863e27de: Status 404 returned error can't find the container with id a7c9fbcb2cdba937cd1c43d8fd0ee6d626ef6b883535e7f7aec2cbb1863e27de
Jul 07 11:11:28.877906 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.877879    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-tt42z"
Jul 07 11:11:28.910289 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.910262    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f9ec8255-2877-4c6a-b248-b50d64978f7a-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-vj4hq\" (UID: \"f9ec8255-2877-4c6a-b248-b50d64978f7a\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-vj4hq"
Jul 07 11:11:28.910526 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.910302    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpp89\" (UniqueName: \"kubernetes.io/projected/0d017aa6-bcf4-4ff7-9161-e477849c7f10-kube-api-access-dpp89\") pod \"network-check-source-989f8b8dc-r6v22\" (UID: \"0d017aa6-bcf4-4ff7-9161-e477849c7f10\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-r6v22"
Jul 07 11:11:28.910526 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.910362    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/63454a74-dfe7-497f-9f3d-0725bc111250-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-lmnhq\" (UID: \"63454a74-dfe7-497f-9f3d-0725bc111250\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-lmnhq"
Jul 07 11:11:28.910526 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.910466    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f9ec8255-2877-4c6a-b248-b50d64978f7a-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-vj4hq\" (UID: \"f9ec8255-2877-4c6a-b248-b50d64978f7a\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-vj4hq"
Jul 07 11:11:28.910526 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.910501    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8kng\" (UniqueName: \"kubernetes.io/projected/63454a74-dfe7-497f-9f3d-0725bc111250-kube-api-access-v8kng\") pod \"cluster-samples-operator-56fcf86bf-lmnhq\" (UID: \"63454a74-dfe7-497f-9f3d-0725bc111250\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-lmnhq"
Jul 07 11:11:28.910732 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:28.910591    2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 07 11:11:28.910732 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:28.910607    2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 07 11:11:28.910732 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:28.910653    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63454a74-dfe7-497f-9f3d-0725bc111250-samples-operator-tls podName:63454a74-dfe7-497f-9f3d-0725bc111250 nodeName:}" failed. No retries permitted until 2026-07-07 11:11:29.41063202 +0000 UTC m=+39.120528405 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/63454a74-dfe7-497f-9f3d-0725bc111250-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-lmnhq" (UID: "63454a74-dfe7-497f-9f3d-0725bc111250") : secret "samples-operator-tls" not found
Jul 07 11:11:28.910732 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:28.910687    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9ec8255-2877-4c6a-b248-b50d64978f7a-networking-console-plugin-cert podName:f9ec8255-2877-4c6a-b248-b50d64978f7a nodeName:}" failed. No retries permitted until 2026-07-07 11:11:29.410670789 +0000 UTC m=+39.120567154 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f9ec8255-2877-4c6a-b248-b50d64978f7a-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-vj4hq" (UID: "f9ec8255-2877-4c6a-b248-b50d64978f7a") : secret "networking-console-plugin-cert" not found
Jul 07 11:11:28.911035 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.911013    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f9ec8255-2877-4c6a-b248-b50d64978f7a-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-vj4hq\" (UID: \"f9ec8255-2877-4c6a-b248-b50d64978f7a\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-vj4hq"
Jul 07 11:11:28.921119 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.921062    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpp89\" (UniqueName: \"kubernetes.io/projected/0d017aa6-bcf4-4ff7-9161-e477849c7f10-kube-api-access-dpp89\") pod \"network-check-source-989f8b8dc-r6v22\" (UID: \"0d017aa6-bcf4-4ff7-9161-e477849c7f10\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-r6v22"
Jul 07 11:11:28.921712 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.921687    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8kng\" (UniqueName: \"kubernetes.io/projected/63454a74-dfe7-497f-9f3d-0725bc111250-kube-api-access-v8kng\") pod \"cluster-samples-operator-56fcf86bf-lmnhq\" (UID: \"63454a74-dfe7-497f-9f3d-0725bc111250\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-lmnhq"
Jul 07 11:11:28.925404 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.925375    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxztq_c20b78c4-cef9-4f47-ac6e-0ac4303eb857/konnectivity-agent/0.log" limit=1024
Jul 07 11:11:28.939185 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.939138    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-r6v22"
Jul 07 11:11:28.981376 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.981261    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6h7pw"]
Jul 07 11:11:28.993346 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:28.993253    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxztq_c20b78c4-cef9-4f47-ac6e-0ac4303eb857/konnectivity-agent/0.log" limit=1024
Jul 07 11:11:29.061362 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:29.061261    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-4v7v8"]
Jul 07 11:11:29.065064 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:11:29.065026    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8c17c90_3e07_4530_8411_d59cef9a1aff.slice/crio-efa94a8cd77c037ac7d1d67a2ec0f280c6feea2589a2645f00911c90ac8f8d93 WatchSource:0}: Error finding container efa94a8cd77c037ac7d1d67a2ec0f280c6feea2589a2645f00911c90ac8f8d93: Status 404 returned error can't find the container with id efa94a8cd77c037ac7d1d67a2ec0f280c6feea2589a2645f00911c90ac8f8d93
Jul 07 11:11:29.091665 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:29.091635    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-4v7v8" event={"ID":"c8c17c90-3e07-4530-8411-d59cef9a1aff","Type":"ContainerStarted","Data":"efa94a8cd77c037ac7d1d67a2ec0f280c6feea2589a2645f00911c90ac8f8d93"}
Jul 07 11:11:29.092785 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:29.092757    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-wffz6" event={"ID":"6387d3da-1290-437c-8fdc-2422d49dac72","Type":"ContainerStarted","Data":"a7c9fbcb2cdba937cd1c43d8fd0ee6d626ef6b883535e7f7aec2cbb1863e27de"}
Jul 07 11:11:29.097134 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:29.097113    2570 generic.go:356] "Generic (PLEG): container finished" podID="09a382c5-57c1-4372-92e4-d24a28620c29" containerID="47d280525d8b6e687d425d1568b4febeee818957c801c546423826a79a7f74ce" exitCode=0
Jul 07 11:11:29.097231 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:29.097214    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gm84h" event={"ID":"09a382c5-57c1-4372-92e4-d24a28620c29","Type":"ContainerDied","Data":"47d280525d8b6e687d425d1568b4febeee818957c801c546423826a79a7f74ce"}
Jul 07 11:11:29.100361 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:29.100309    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6h7pw" event={"ID":"7fba6280-ffbe-4a71-98f3-3432b0584697","Type":"ContainerStarted","Data":"8a3045ba517aeb6c4da4db06e1ebeebd3a268b9a354cf81e4f196d66836dad62"}
Jul 07 11:11:29.129070 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:29.129001    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-95lgm"]
Jul 07 11:11:29.131819 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:11:29.131790    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod192f418c_f6e6_4775_876f_ef3d4f11b265.slice/crio-878169d53f206878d8974940f50e9379f9c36b837a3c5ea03fe1bab640d8dea4 WatchSource:0}: Error finding container 878169d53f206878d8974940f50e9379f9c36b837a3c5ea03fe1bab640d8dea4: Status 404 returned error can't find the container with id 878169d53f206878d8974940f50e9379f9c36b837a3c5ea03fe1bab640d8dea4
Jul 07 11:11:29.154440 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:29.154411    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-tt42z"]
Jul 07 11:11:29.157510 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:11:29.157482    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62ea12c1_c37b_4e29_9700_bd826f004e51.slice/crio-86338e76bf9e86b05801e68ffb05fb39a1236fc464281fdbbf82feae33b15f30 WatchSource:0}: Error finding container 86338e76bf9e86b05801e68ffb05fb39a1236fc464281fdbbf82feae33b15f30: Status 404 returned error can't find the container with id 86338e76bf9e86b05801e68ffb05fb39a1236fc464281fdbbf82feae33b15f30
Jul 07 11:11:29.174230 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:29.174161    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-r6v22"]
Jul 07 11:11:29.180501 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:11:29.180473    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d017aa6_bcf4_4ff7_9161_e477849c7f10.slice/crio-2a876fbf36c47ace3564207cb10f981404d8035038713fd35a051bd9242114f1 WatchSource:0}: Error finding container 2a876fbf36c47ace3564207cb10f981404d8035038713fd35a051bd9242114f1: Status 404 returned error can't find the container with id 2a876fbf36c47ace3564207cb10f981404d8035038713fd35a051bd9242114f1
Jul 07 11:11:29.213200 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:29.213157    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91203aa4-b720-4390-a2c0-5c553ffd79f8-metrics-certs\") pod \"router-default-5dc4cf6785-bflm8\" (UID: \"91203aa4-b720-4390-a2c0-5c553ffd79f8\") " pod="openshift-ingress/router-default-5dc4cf6785-bflm8"
Jul 07 11:11:29.213308 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:29.213213    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91203aa4-b720-4390-a2c0-5c553ffd79f8-service-ca-bundle\") pod \"router-default-5dc4cf6785-bflm8\" (UID: \"91203aa4-b720-4390-a2c0-5c553ffd79f8\") " pod="openshift-ingress/router-default-5dc4cf6785-bflm8"
Jul 07 11:11:29.213402 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:29.213307    2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 07 11:11:29.213402 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:29.213359    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/91203aa4-b720-4390-a2c0-5c553ffd79f8-service-ca-bundle podName:91203aa4-b720-4390-a2c0-5c553ffd79f8 nodeName:}" failed. No retries permitted until 2026-07-07 11:11:30.213314594 +0000 UTC m=+39.923210974 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/91203aa4-b720-4390-a2c0-5c553ffd79f8-service-ca-bundle") pod "router-default-5dc4cf6785-bflm8" (UID: "91203aa4-b720-4390-a2c0-5c553ffd79f8") : configmap references non-existent config key: service-ca.crt
Jul 07 11:11:29.213712 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:29.213697    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91203aa4-b720-4390-a2c0-5c553ffd79f8-metrics-certs podName:91203aa4-b720-4390-a2c0-5c553ffd79f8 nodeName:}" failed. No retries permitted until 2026-07-07 11:11:30.213676481 +0000 UTC m=+39.923572852 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/91203aa4-b720-4390-a2c0-5c553ffd79f8-metrics-certs") pod "router-default-5dc4cf6785-bflm8" (UID: "91203aa4-b720-4390-a2c0-5c553ffd79f8") : secret "router-metrics-certs-default" not found
Jul 07 11:11:29.315108 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:29.315079    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/df6d16a6-2593-4477-ae7a-eca848817f34-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-9pdtd\" (UID: \"df6d16a6-2593-4477-ae7a-eca848817f34\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-9pdtd"
Jul 07 11:11:29.315298 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:29.315284    2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 07 11:11:29.315626 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:29.315364    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df6d16a6-2593-4477-ae7a-eca848817f34-cluster-monitoring-operator-tls podName:df6d16a6-2593-4477-ae7a-eca848817f34 nodeName:}" failed. No retries permitted until 2026-07-07 11:11:30.315343415 +0000 UTC m=+40.025239802 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/df6d16a6-2593-4477-ae7a-eca848817f34-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-9pdtd" (UID: "df6d16a6-2593-4477-ae7a-eca848817f34") : secret "cluster-monitoring-operator-tls" not found
Jul 07 11:11:29.415660 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:29.415580    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/63454a74-dfe7-497f-9f3d-0725bc111250-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-lmnhq\" (UID: \"63454a74-dfe7-497f-9f3d-0725bc111250\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-lmnhq"
Jul 07 11:11:29.416405 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:29.415748    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f9ec8255-2877-4c6a-b248-b50d64978f7a-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-vj4hq\" (UID: \"f9ec8255-2877-4c6a-b248-b50d64978f7a\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-vj4hq"
Jul 07 11:11:29.416405 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:29.415767    2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 07 11:11:29.416405 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:29.415840    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63454a74-dfe7-497f-9f3d-0725bc111250-samples-operator-tls podName:63454a74-dfe7-497f-9f3d-0725bc111250 nodeName:}" failed. No retries permitted until 2026-07-07 11:11:30.415818129 +0000 UTC m=+40.125714498 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/63454a74-dfe7-497f-9f3d-0725bc111250-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-lmnhq" (UID: "63454a74-dfe7-497f-9f3d-0725bc111250") : secret "samples-operator-tls" not found
Jul 07 11:11:29.416405 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:29.415879    2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 07 11:11:29.416405 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:29.415931    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9ec8255-2877-4c6a-b248-b50d64978f7a-networking-console-plugin-cert podName:f9ec8255-2877-4c6a-b248-b50d64978f7a nodeName:}" failed. No retries permitted until 2026-07-07 11:11:30.41591565 +0000 UTC m=+40.125812017 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f9ec8255-2877-4c6a-b248-b50d64978f7a-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-vj4hq" (UID: "f9ec8255-2877-4c6a-b248-b50d64978f7a") : secret "networking-console-plugin-cert" not found
Jul 07 11:11:29.924977 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:29.924560    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/daf8c8f2-2b41-4800-a056-1f1b1d9714b7-metrics-tls\") pod \"dns-default-64n4d\" (UID: \"daf8c8f2-2b41-4800-a056-1f1b1d9714b7\") " pod="openshift-dns/dns-default-64n4d"
Jul 07 11:11:29.924977 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:29.924623    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc075655-13c0-4026-9ed2-8c4cc2a71dab-registry-tls\") pod \"image-registry-9bb4d966c-zfwhd\" (UID: \"cc075655-13c0-4026-9ed2-8c4cc2a71dab\") " pod="openshift-image-registry/image-registry-9bb4d966c-zfwhd"
Jul 07 11:11:29.924977 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:29.924810    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 11:11:29.924977 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:29.924828    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9bb4d966c-zfwhd: secret "image-registry-tls" not found
Jul 07 11:11:29.924977 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:29.924890    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 11:11:29.924977 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:29.924919    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cc075655-13c0-4026-9ed2-8c4cc2a71dab-registry-tls podName:cc075655-13c0-4026-9ed2-8c4cc2a71dab nodeName:}" failed. No retries permitted until 2026-07-07 11:11:33.924873859 +0000 UTC m=+43.634770240 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cc075655-13c0-4026-9ed2-8c4cc2a71dab-registry-tls") pod "image-registry-9bb4d966c-zfwhd" (UID: "cc075655-13c0-4026-9ed2-8c4cc2a71dab") : secret "image-registry-tls" not found
Jul 07 11:11:29.924977 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:29.924938    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/daf8c8f2-2b41-4800-a056-1f1b1d9714b7-metrics-tls podName:daf8c8f2-2b41-4800-a056-1f1b1d9714b7 nodeName:}" failed. No retries permitted until 2026-07-07 11:11:33.924928145 +0000 UTC m=+43.634824513 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/daf8c8f2-2b41-4800-a056-1f1b1d9714b7-metrics-tls") pod "dns-default-64n4d" (UID: "daf8c8f2-2b41-4800-a056-1f1b1d9714b7") : secret "dns-default-metrics-tls" not found
Jul 07 11:11:30.109024 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:30.108934    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-95lgm" event={"ID":"192f418c-f6e6-4775-876f-ef3d4f11b265","Type":"ContainerStarted","Data":"878169d53f206878d8974940f50e9379f9c36b837a3c5ea03fe1bab640d8dea4"}
Jul 07 11:11:30.121904 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:30.120952    2570 generic.go:356] "Generic (PLEG): container finished" podID="09a382c5-57c1-4372-92e4-d24a28620c29" containerID="bcff59ba4b582f8c25dbd0a9148ee80ff2ba0977995de13995c3d80ded5a386c" exitCode=0
Jul 07 11:11:30.121904 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:30.121048    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gm84h" event={"ID":"09a382c5-57c1-4372-92e4-d24a28620c29","Type":"ContainerDied","Data":"bcff59ba4b582f8c25dbd0a9148ee80ff2ba0977995de13995c3d80ded5a386c"}
Jul 07 11:11:30.125938 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:30.125906    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-r6v22" event={"ID":"0d017aa6-bcf4-4ff7-9161-e477849c7f10","Type":"ContainerStarted","Data":"2a876fbf36c47ace3564207cb10f981404d8035038713fd35a051bd9242114f1"}
Jul 07 11:11:30.129556 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:30.129491    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-tt42z" event={"ID":"62ea12c1-c37b-4e29-9700-bd826f004e51","Type":"ContainerStarted","Data":"86338e76bf9e86b05801e68ffb05fb39a1236fc464281fdbbf82feae33b15f30"}
Jul 07 11:11:30.227030 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:30.226988    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91203aa4-b720-4390-a2c0-5c553ffd79f8-metrics-certs\") pod \"router-default-5dc4cf6785-bflm8\" (UID: \"91203aa4-b720-4390-a2c0-5c553ffd79f8\") " pod="openshift-ingress/router-default-5dc4cf6785-bflm8"
Jul 07 11:11:30.227192 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:30.227051    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91203aa4-b720-4390-a2c0-5c553ffd79f8-service-ca-bundle\") pod \"router-default-5dc4cf6785-bflm8\" (UID: \"91203aa4-b720-4390-a2c0-5c553ffd79f8\") " pod="openshift-ingress/router-default-5dc4cf6785-bflm8"
Jul 07 11:11:30.227684 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:30.227661    2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 07 11:11:30.227791 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:30.227741    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91203aa4-b720-4390-a2c0-5c553ffd79f8-metrics-certs podName:91203aa4-b720-4390-a2c0-5c553ffd79f8 nodeName:}" failed. No retries permitted until 2026-07-07 11:11:32.227720452 +0000 UTC m=+41.937616834 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/91203aa4-b720-4390-a2c0-5c553ffd79f8-metrics-certs") pod "router-default-5dc4cf6785-bflm8" (UID: "91203aa4-b720-4390-a2c0-5c553ffd79f8") : secret "router-metrics-certs-default" not found
Jul 07 11:11:30.228187 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:30.228167    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/91203aa4-b720-4390-a2c0-5c553ffd79f8-service-ca-bundle podName:91203aa4-b720-4390-a2c0-5c553ffd79f8 nodeName:}" failed. No retries permitted until 2026-07-07 11:11:32.228152068 +0000 UTC m=+41.938048438 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/91203aa4-b720-4390-a2c0-5c553ffd79f8-service-ca-bundle") pod "router-default-5dc4cf6785-bflm8" (UID: "91203aa4-b720-4390-a2c0-5c553ffd79f8") : configmap references non-existent config key: service-ca.crt
Jul 07 11:11:30.328749 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:30.327908    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/df6d16a6-2593-4477-ae7a-eca848817f34-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-9pdtd\" (UID: \"df6d16a6-2593-4477-ae7a-eca848817f34\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-9pdtd"
Jul 07 11:11:30.328749 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:30.328281    2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 07 11:11:30.328749 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:30.328369    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df6d16a6-2593-4477-ae7a-eca848817f34-cluster-monitoring-operator-tls podName:df6d16a6-2593-4477-ae7a-eca848817f34 nodeName:}" failed. No retries permitted until 2026-07-07 11:11:32.328347633 +0000 UTC m=+42.038244000 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/df6d16a6-2593-4477-ae7a-eca848817f34-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-9pdtd" (UID: "df6d16a6-2593-4477-ae7a-eca848817f34") : secret "cluster-monitoring-operator-tls" not found
Jul 07 11:11:30.430581 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:30.429557    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/63454a74-dfe7-497f-9f3d-0725bc111250-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-lmnhq\" (UID: \"63454a74-dfe7-497f-9f3d-0725bc111250\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-lmnhq"
Jul 07 11:11:30.430581 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:30.429727    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f9ec8255-2877-4c6a-b248-b50d64978f7a-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-vj4hq\" (UID: \"f9ec8255-2877-4c6a-b248-b50d64978f7a\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-vj4hq"
Jul 07 11:11:30.430581 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:30.429907    2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 07 11:11:30.430581 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:30.429971    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9ec8255-2877-4c6a-b248-b50d64978f7a-networking-console-plugin-cert podName:f9ec8255-2877-4c6a-b248-b50d64978f7a nodeName:}" failed. No retries permitted until 2026-07-07 11:11:32.429951923 +0000 UTC m=+42.139848301 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f9ec8255-2877-4c6a-b248-b50d64978f7a-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-vj4hq" (UID: "f9ec8255-2877-4c6a-b248-b50d64978f7a") : secret "networking-console-plugin-cert" not found
Jul 07 11:11:30.430581 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:30.430131    2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 07 11:11:30.430581 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:30.430167    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63454a74-dfe7-497f-9f3d-0725bc111250-samples-operator-tls podName:63454a74-dfe7-497f-9f3d-0725bc111250 nodeName:}" failed. No retries permitted until 2026-07-07 11:11:32.430154673 +0000 UTC m=+42.140051041 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/63454a74-dfe7-497f-9f3d-0725bc111250-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-lmnhq" (UID: "63454a74-dfe7-497f-9f3d-0725bc111250") : secret "samples-operator-tls" not found
Jul 07 11:11:31.142425 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:31.141450    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gm84h" event={"ID":"09a382c5-57c1-4372-92e4-d24a28620c29","Type":"ContainerStarted","Data":"bfbde45c3f292e4018180123bc36d3900a0d918af1128f0323c7d89918a4e288"}
Jul 07 11:11:31.169740 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:31.169142    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-gm84h" podStartSLOduration=9.791535047 podStartE2EDuration="40.169119343s" totalImagesPullingTime="30.377584296s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:10:51 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:11:31.166929166 +0000 UTC m=+40.876825554" watchObservedRunningTime="2026-07-07 11:11:31.169119343 +0000 UTC m=+40.879015733"
Jul 07 11:11:31.292067 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:31.292032    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxztq_c20b78c4-cef9-4f47-ac6e-0ac4303eb857/konnectivity-agent/0.log" limit=1024
Jul 07 11:11:31.369058 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:31.369018    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxztq_c20b78c4-cef9-4f47-ac6e-0ac4303eb857/konnectivity-agent/0.log" limit=1024
Jul 07 11:11:31.591300 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:31.591267    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxztq_c20b78c4-cef9-4f47-ac6e-0ac4303eb857/konnectivity-agent/0.log" limit=1024
Jul 07 11:11:31.683599 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:31.683561    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxztq_c20b78c4-cef9-4f47-ac6e-0ac4303eb857/konnectivity-agent/0.log" limit=1024
Jul 07 11:11:32.124451 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:32.124414    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxztq_c20b78c4-cef9-4f47-ac6e-0ac4303eb857/konnectivity-agent/0.log" limit=1024
Jul 07 11:11:32.221800 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:32.221739    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxztq_c20b78c4-cef9-4f47-ac6e-0ac4303eb857/konnectivity-agent/0.log" limit=1024
Jul 07 11:11:32.246519 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:32.246482    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91203aa4-b720-4390-a2c0-5c553ffd79f8-service-ca-bundle\") pod \"router-default-5dc4cf6785-bflm8\" (UID: \"91203aa4-b720-4390-a2c0-5c553ffd79f8\") " pod="openshift-ingress/router-default-5dc4cf6785-bflm8"
Jul 07 11:11:32.246775 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:32.246751    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/91203aa4-b720-4390-a2c0-5c553ffd79f8-service-ca-bundle podName:91203aa4-b720-4390-a2c0-5c553ffd79f8 nodeName:}" failed. No retries permitted until 2026-07-07 11:11:36.246709539 +0000 UTC m=+45.956605909 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/91203aa4-b720-4390-a2c0-5c553ffd79f8-service-ca-bundle") pod "router-default-5dc4cf6785-bflm8" (UID: "91203aa4-b720-4390-a2c0-5c553ffd79f8") : configmap references non-existent config key: service-ca.crt
Jul 07 11:11:32.246915 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:32.246890    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91203aa4-b720-4390-a2c0-5c553ffd79f8-metrics-certs\") pod \"router-default-5dc4cf6785-bflm8\" (UID: \"91203aa4-b720-4390-a2c0-5c553ffd79f8\") " pod="openshift-ingress/router-default-5dc4cf6785-bflm8"
Jul 07 11:11:32.247043 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:32.247009    2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 07 11:11:32.247106 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:32.247094    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91203aa4-b720-4390-a2c0-5c553ffd79f8-metrics-certs podName:91203aa4-b720-4390-a2c0-5c553ffd79f8 nodeName:}" failed. No retries permitted until 2026-07-07 11:11:36.247074344 +0000 UTC m=+45.956970728 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/91203aa4-b720-4390-a2c0-5c553ffd79f8-metrics-certs") pod "router-default-5dc4cf6785-bflm8" (UID: "91203aa4-b720-4390-a2c0-5c553ffd79f8") : secret "router-metrics-certs-default" not found
Jul 07 11:11:32.347890 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:32.347848    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/df6d16a6-2593-4477-ae7a-eca848817f34-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-9pdtd\" (UID: \"df6d16a6-2593-4477-ae7a-eca848817f34\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-9pdtd"
Jul 07 11:11:32.348071 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:32.348008    2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 07 11:11:32.348137 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:32.348094    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df6d16a6-2593-4477-ae7a-eca848817f34-cluster-monitoring-operator-tls podName:df6d16a6-2593-4477-ae7a-eca848817f34 nodeName:}" failed. No retries permitted until 2026-07-07 11:11:36.348071677 +0000 UTC m=+46.057968043 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/df6d16a6-2593-4477-ae7a-eca848817f34-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-9pdtd" (UID: "df6d16a6-2593-4477-ae7a-eca848817f34") : secret "cluster-monitoring-operator-tls" not found
Jul 07 11:11:32.448674 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:32.448584    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/63454a74-dfe7-497f-9f3d-0725bc111250-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-lmnhq\" (UID: \"63454a74-dfe7-497f-9f3d-0725bc111250\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-lmnhq"
Jul 07 11:11:32.448854 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:32.448727    2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 07 11:11:32.448854 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:32.448789    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63454a74-dfe7-497f-9f3d-0725bc111250-samples-operator-tls podName:63454a74-dfe7-497f-9f3d-0725bc111250 nodeName:}" failed. No retries permitted until 2026-07-07 11:11:36.44877402 +0000 UTC m=+46.158670386 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/63454a74-dfe7-497f-9f3d-0725bc111250-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-lmnhq" (UID: "63454a74-dfe7-497f-9f3d-0725bc111250") : secret "samples-operator-tls" not found
Jul 07 11:11:32.448854 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:32.448734    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f9ec8255-2877-4c6a-b248-b50d64978f7a-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-vj4hq\" (UID: \"f9ec8255-2877-4c6a-b248-b50d64978f7a\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-vj4hq"
Jul 07 11:11:32.448854 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:32.448816    2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 07 11:11:32.449055 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:32.448873    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9ec8255-2877-4c6a-b248-b50d64978f7a-networking-console-plugin-cert podName:f9ec8255-2877-4c6a-b248-b50d64978f7a nodeName:}" failed. No retries permitted until 2026-07-07 11:11:36.448857582 +0000 UTC m=+46.158753947 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f9ec8255-2877-4c6a-b248-b50d64978f7a-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-vj4hq" (UID: "f9ec8255-2877-4c6a-b248-b50d64978f7a") : secret "networking-console-plugin-cert" not found
Jul 07 11:11:33.962385 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:33.962171    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/daf8c8f2-2b41-4800-a056-1f1b1d9714b7-metrics-tls\") pod \"dns-default-64n4d\" (UID: \"daf8c8f2-2b41-4800-a056-1f1b1d9714b7\") " pod="openshift-dns/dns-default-64n4d"
Jul 07 11:11:33.962385 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:33.962228    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc075655-13c0-4026-9ed2-8c4cc2a71dab-registry-tls\") pod \"image-registry-9bb4d966c-zfwhd\" (UID: \"cc075655-13c0-4026-9ed2-8c4cc2a71dab\") " pod="openshift-image-registry/image-registry-9bb4d966c-zfwhd"
Jul 07 11:11:33.962385 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:33.962377    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 11:11:33.962914 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:33.962406    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 11:11:33.962914 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:33.962429    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9bb4d966c-zfwhd: secret "image-registry-tls" not found
Jul 07 11:11:33.962914 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:33.962471    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/daf8c8f2-2b41-4800-a056-1f1b1d9714b7-metrics-tls podName:daf8c8f2-2b41-4800-a056-1f1b1d9714b7 nodeName:}" failed. No retries permitted until 2026-07-07 11:11:41.962446766 +0000 UTC m=+51.672343132 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/daf8c8f2-2b41-4800-a056-1f1b1d9714b7-metrics-tls") pod "dns-default-64n4d" (UID: "daf8c8f2-2b41-4800-a056-1f1b1d9714b7") : secret "dns-default-metrics-tls" not found
Jul 07 11:11:33.962914 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:33.962495    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cc075655-13c0-4026-9ed2-8c4cc2a71dab-registry-tls podName:cc075655-13c0-4026-9ed2-8c4cc2a71dab nodeName:}" failed. No retries permitted until 2026-07-07 11:11:41.962484708 +0000 UTC m=+51.672381081 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cc075655-13c0-4026-9ed2-8c4cc2a71dab-registry-tls") pod "image-registry-9bb4d966c-zfwhd" (UID: "cc075655-13c0-4026-9ed2-8c4cc2a71dab") : secret "image-registry-tls" not found
Jul 07 11:11:35.443193 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:35.443160    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxztq_c20b78c4-cef9-4f47-ac6e-0ac4303eb857/konnectivity-agent/0.log" limit=1024
Jul 07 11:11:35.561121 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:35.561089    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxztq_c20b78c4-cef9-4f47-ac6e-0ac4303eb857/konnectivity-agent/0.log" limit=1024
Jul 07 11:11:36.288563 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:36.288520    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91203aa4-b720-4390-a2c0-5c553ffd79f8-metrics-certs\") pod \"router-default-5dc4cf6785-bflm8\" (UID: \"91203aa4-b720-4390-a2c0-5c553ffd79f8\") " pod="openshift-ingress/router-default-5dc4cf6785-bflm8"
Jul 07 11:11:36.288563 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:36.288567    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91203aa4-b720-4390-a2c0-5c553ffd79f8-service-ca-bundle\") pod \"router-default-5dc4cf6785-bflm8\" (UID: \"91203aa4-b720-4390-a2c0-5c553ffd79f8\") " pod="openshift-ingress/router-default-5dc4cf6785-bflm8"
Jul 07 11:11:36.288793 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:36.288688    2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 07 11:11:36.288793 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:36.288762    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91203aa4-b720-4390-a2c0-5c553ffd79f8-metrics-certs podName:91203aa4-b720-4390-a2c0-5c553ffd79f8 nodeName:}" failed. No retries permitted until 2026-07-07 11:11:44.288742478 +0000 UTC m=+53.998638858 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/91203aa4-b720-4390-a2c0-5c553ffd79f8-metrics-certs") pod "router-default-5dc4cf6785-bflm8" (UID: "91203aa4-b720-4390-a2c0-5c553ffd79f8") : secret "router-metrics-certs-default" not found
Jul 07 11:11:36.288793 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:36.288782    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/91203aa4-b720-4390-a2c0-5c553ffd79f8-service-ca-bundle podName:91203aa4-b720-4390-a2c0-5c553ffd79f8 nodeName:}" failed. No retries permitted until 2026-07-07 11:11:44.288772741 +0000 UTC m=+53.998669109 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/91203aa4-b720-4390-a2c0-5c553ffd79f8-service-ca-bundle") pod "router-default-5dc4cf6785-bflm8" (UID: "91203aa4-b720-4390-a2c0-5c553ffd79f8") : configmap references non-existent config key: service-ca.crt
Jul 07 11:11:36.389008 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:36.388967    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/df6d16a6-2593-4477-ae7a-eca848817f34-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-9pdtd\" (UID: \"df6d16a6-2593-4477-ae7a-eca848817f34\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-9pdtd"
Jul 07 11:11:36.389192 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:36.389138    2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 07 11:11:36.389249 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:36.389224    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df6d16a6-2593-4477-ae7a-eca848817f34-cluster-monitoring-operator-tls podName:df6d16a6-2593-4477-ae7a-eca848817f34 nodeName:}" failed. No retries permitted until 2026-07-07 11:11:44.389202744 +0000 UTC m=+54.099099111 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/df6d16a6-2593-4477-ae7a-eca848817f34-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-9pdtd" (UID: "df6d16a6-2593-4477-ae7a-eca848817f34") : secret "cluster-monitoring-operator-tls" not found
Jul 07 11:11:36.425837 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:36.425785    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxztq_c20b78c4-cef9-4f47-ac6e-0ac4303eb857/konnectivity-agent/0.log" limit=1024
Jul 07 11:11:36.490385 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:36.490339    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f9ec8255-2877-4c6a-b248-b50d64978f7a-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-vj4hq\" (UID: \"f9ec8255-2877-4c6a-b248-b50d64978f7a\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-vj4hq"
Jul 07 11:11:36.490747 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:36.490446    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/63454a74-dfe7-497f-9f3d-0725bc111250-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-lmnhq\" (UID: \"63454a74-dfe7-497f-9f3d-0725bc111250\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-lmnhq"
Jul 07 11:11:36.490747 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:36.490491    2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 07 11:11:36.490747 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:36.490552    2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 07 11:11:36.490747 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:36.490559    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9ec8255-2877-4c6a-b248-b50d64978f7a-networking-console-plugin-cert podName:f9ec8255-2877-4c6a-b248-b50d64978f7a nodeName:}" failed. No retries permitted until 2026-07-07 11:11:44.490540503 +0000 UTC m=+54.200436888 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f9ec8255-2877-4c6a-b248-b50d64978f7a-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-vj4hq" (UID: "f9ec8255-2877-4c6a-b248-b50d64978f7a") : secret "networking-console-plugin-cert" not found
Jul 07 11:11:36.490747 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:36.490586    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63454a74-dfe7-497f-9f3d-0725bc111250-samples-operator-tls podName:63454a74-dfe7-497f-9f3d-0725bc111250 nodeName:}" failed. No retries permitted until 2026-07-07 11:11:44.49057566 +0000 UTC m=+54.200472025 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/63454a74-dfe7-497f-9f3d-0725bc111250-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-lmnhq" (UID: "63454a74-dfe7-497f-9f3d-0725bc111250") : secret "samples-operator-tls" not found
Jul 07 11:11:36.494007 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:36.493976    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxztq_c20b78c4-cef9-4f47-ac6e-0ac4303eb857/konnectivity-agent/0.log" limit=1024
Jul 07 11:11:37.159388 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:37.159366    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4v7v8_c8c17c90-3e07-4530-8411-d59cef9a1aff/console-operator/0.log"
Jul 07 11:11:37.159514 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:37.159406    2570 generic.go:356] "Generic (PLEG): container finished" podID="c8c17c90-3e07-4530-8411-d59cef9a1aff" containerID="7fa52da130bdfdf6ec86ca00321110e8e73ed9107562d8fb338e86a6275ffef4" exitCode=255
Jul 07 11:11:37.159514 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:37.159484    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-4v7v8" event={"ID":"c8c17c90-3e07-4530-8411-d59cef9a1aff","Type":"ContainerDied","Data":"7fa52da130bdfdf6ec86ca00321110e8e73ed9107562d8fb338e86a6275ffef4"}
Jul 07 11:11:37.159775 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:37.159743    2570 scope.go:117] "RemoveContainer" containerID="7fa52da130bdfdf6ec86ca00321110e8e73ed9107562d8fb338e86a6275ffef4"
Jul 07 11:11:37.161216 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:37.161193    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-wffz6" event={"ID":"6387d3da-1290-437c-8fdc-2422d49dac72","Type":"ContainerStarted","Data":"001aeae498a26940179e64df14dc5a0862488a4c82b9897c7570f19542670d31"}
Jul 07 11:11:37.162596 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:37.162576    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wlbnq" event={"ID":"3e1e8d03-ef0d-49d9-a760-64039277242c","Type":"ContainerStarted","Data":"c204f64c4294262263a77568d1c5294d9995fb424a9b6afcf730c8c7c2a59cc2"}
Jul 07 11:11:37.198102 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:37.197410    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-wlbnq" podStartSLOduration=1.119109301 podStartE2EDuration="11.197390512s" totalImagesPullingTime="10.078281211s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:11:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:11:37.194862835 +0000 UTC m=+46.904759224" watchObservedRunningTime="2026-07-07 11:11:37.197390512 +0000 UTC m=+46.907286900"
Jul 07 11:11:37.216661 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:37.215693    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-wffz6" podStartSLOduration=1.361395486 podStartE2EDuration="9.215672103s" totalImagesPullingTime="7.854276617s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:11:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:11:37.21367457 +0000 UTC m=+46.923570957" watchObservedRunningTime="2026-07-07 11:11:37.215672103 +0000 UTC m=+46.925568491"
Jul 07 11:11:37.654676 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:37.654638    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-8kjb7"]
Jul 07 11:11:37.686402 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:37.686367    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8kjb7"]
Jul 07 11:11:37.686570 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:37.686469    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8kjb7"
Jul 07 11:11:37.689433 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:37.689404    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 07 11:11:37.689704 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:37.689664    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxztq_c20b78c4-cef9-4f47-ac6e-0ac4303eb857/konnectivity-agent/0.log" limit=1024
Jul 07 11:11:37.689825 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:37.689791    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 07 11:11:37.689825 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:37.689797    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tnjrv\""
Jul 07 11:11:37.765184 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:37.765149    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxztq_c20b78c4-cef9-4f47-ac6e-0ac4303eb857/konnectivity-agent/0.log" limit=1024
Jul 07 11:11:37.801671 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:37.801638    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cc937ff9-effb-4a24-a888-06fc5f4e5fe6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8kjb7\" (UID: \"cc937ff9-effb-4a24-a888-06fc5f4e5fe6\") " pod="openshift-insights/insights-runtime-extractor-8kjb7"
Jul 07 11:11:37.801847 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:37.801683    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cc937ff9-effb-4a24-a888-06fc5f4e5fe6-data-volume\") pod \"insights-runtime-extractor-8kjb7\" (UID: \"cc937ff9-effb-4a24-a888-06fc5f4e5fe6\") " pod="openshift-insights/insights-runtime-extractor-8kjb7"
Jul 07 11:11:37.801847 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:37.801756    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cc937ff9-effb-4a24-a888-06fc5f4e5fe6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8kjb7\" (UID: \"cc937ff9-effb-4a24-a888-06fc5f4e5fe6\") " pod="openshift-insights/insights-runtime-extractor-8kjb7"
Jul 07 11:11:37.801847 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:37.801783    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qxxs\" (UniqueName: \"kubernetes.io/projected/cc937ff9-effb-4a24-a888-06fc5f4e5fe6-kube-api-access-9qxxs\") pod \"insights-runtime-extractor-8kjb7\" (UID: \"cc937ff9-effb-4a24-a888-06fc5f4e5fe6\") " pod="openshift-insights/insights-runtime-extractor-8kjb7"
Jul 07 11:11:37.801991 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:37.801916    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cc937ff9-effb-4a24-a888-06fc5f4e5fe6-crio-socket\") pod \"insights-runtime-extractor-8kjb7\" (UID: \"cc937ff9-effb-4a24-a888-06fc5f4e5fe6\") " pod="openshift-insights/insights-runtime-extractor-8kjb7"
Jul 07 11:11:37.903166 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:37.903131    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cc937ff9-effb-4a24-a888-06fc5f4e5fe6-data-volume\") pod \"insights-runtime-extractor-8kjb7\" (UID: \"cc937ff9-effb-4a24-a888-06fc5f4e5fe6\") " pod="openshift-insights/insights-runtime-extractor-8kjb7"
Jul 07 11:11:37.903338 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:37.903233    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cc937ff9-effb-4a24-a888-06fc5f4e5fe6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8kjb7\" (UID: \"cc937ff9-effb-4a24-a888-06fc5f4e5fe6\") " pod="openshift-insights/insights-runtime-extractor-8kjb7"
Jul 07 11:11:37.903338 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:37.903259    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qxxs\" (UniqueName: \"kubernetes.io/projected/cc937ff9-effb-4a24-a888-06fc5f4e5fe6-kube-api-access-9qxxs\") pod \"insights-runtime-extractor-8kjb7\" (UID: \"cc937ff9-effb-4a24-a888-06fc5f4e5fe6\") " pod="openshift-insights/insights-runtime-extractor-8kjb7"
Jul 07 11:11:37.903338 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:37.903306    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cc937ff9-effb-4a24-a888-06fc5f4e5fe6-crio-socket\") pod \"insights-runtime-extractor-8kjb7\" (UID: \"cc937ff9-effb-4a24-a888-06fc5f4e5fe6\") " pod="openshift-insights/insights-runtime-extractor-8kjb7"
Jul 07 11:11:37.903473 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:37.903407    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cc937ff9-effb-4a24-a888-06fc5f4e5fe6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8kjb7\" (UID: \"cc937ff9-effb-4a24-a888-06fc5f4e5fe6\") " pod="openshift-insights/insights-runtime-extractor-8kjb7"
Jul 07 11:11:37.903473 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:37.903411    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 11:11:37.903562 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:37.903476    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc937ff9-effb-4a24-a888-06fc5f4e5fe6-insights-runtime-extractor-tls podName:cc937ff9-effb-4a24-a888-06fc5f4e5fe6 nodeName:}" failed. No retries permitted until 2026-07-07 11:11:38.403461366 +0000 UTC m=+48.113357731 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cc937ff9-effb-4a24-a888-06fc5f4e5fe6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8kjb7" (UID: "cc937ff9-effb-4a24-a888-06fc5f4e5fe6") : secret "insights-runtime-extractor-tls" not found
Jul 07 11:11:37.903562 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:37.903522    2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 11:11:37.903674 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:37.903564    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cc937ff9-effb-4a24-a888-06fc5f4e5fe6-kube-rbac-proxy-cm podName:cc937ff9-effb-4a24-a888-06fc5f4e5fe6 nodeName:}" failed. No retries permitted until 2026-07-07 11:11:38.403551471 +0000 UTC m=+48.113447841 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cc937ff9-effb-4a24-a888-06fc5f4e5fe6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8kjb7" (UID: "cc937ff9-effb-4a24-a888-06fc5f4e5fe6") : configmap "kube-rbac-proxy" not found
Jul 07 11:11:37.903674 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:37.903587    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cc937ff9-effb-4a24-a888-06fc5f4e5fe6-data-volume\") pod \"insights-runtime-extractor-8kjb7\" (UID: \"cc937ff9-effb-4a24-a888-06fc5f4e5fe6\") " pod="openshift-insights/insights-runtime-extractor-8kjb7"
Jul 07 11:11:37.903674 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:37.903605    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cc937ff9-effb-4a24-a888-06fc5f4e5fe6-crio-socket\") pod \"insights-runtime-extractor-8kjb7\" (UID: \"cc937ff9-effb-4a24-a888-06fc5f4e5fe6\") " pod="openshift-insights/insights-runtime-extractor-8kjb7"
Jul 07 11:11:37.916040 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:37.915975    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qxxs\" (UniqueName: \"kubernetes.io/projected/cc937ff9-effb-4a24-a888-06fc5f4e5fe6-kube-api-access-9qxxs\") pod \"insights-runtime-extractor-8kjb7\" (UID: \"cc937ff9-effb-4a24-a888-06fc5f4e5fe6\") " pod="openshift-insights/insights-runtime-extractor-8kjb7"
Jul 07 11:11:38.168857 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:38.168763    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-r6v22" event={"ID":"0d017aa6-bcf4-4ff7-9161-e477849c7f10","Type":"ContainerStarted","Data":"f7d37a85c41cefec02ae1cb5080685c12aac7ae458a0875b8374cb61b55b3e36"}
Jul 07 11:11:38.170442 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:38.170414    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-tt42z" event={"ID":"62ea12c1-c37b-4e29-9700-bd826f004e51","Type":"ContainerStarted","Data":"5546b0670b8bb72aa26ced08641a5e1bede2432ab57ab08cd8e3d24083addcb2"}
Jul 07 11:11:38.171934 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:38.171904    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6h7pw" event={"ID":"7fba6280-ffbe-4a71-98f3-3432b0584697","Type":"ContainerStarted","Data":"052701c116eb9b9b1cec2e45cd595c994407f937caa13282328f227cf88329d9"}
Jul 07 11:11:38.173642 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:38.173622    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4v7v8_c8c17c90-3e07-4530-8411-d59cef9a1aff/console-operator/1.log"
Jul 07 11:11:38.174100 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:38.174080    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4v7v8_c8c17c90-3e07-4530-8411-d59cef9a1aff/console-operator/0.log"
Jul 07 11:11:38.174190 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:38.174114    2570 generic.go:356] "Generic (PLEG): container finished" podID="c8c17c90-3e07-4530-8411-d59cef9a1aff" containerID="a1299409496a2b71caab3591acf44a47769e95b5aed257fb6bfb946c1f2d419c" exitCode=255
Jul 07 11:11:38.174190 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:38.174176    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-4v7v8" event={"ID":"c8c17c90-3e07-4530-8411-d59cef9a1aff","Type":"ContainerDied","Data":"a1299409496a2b71caab3591acf44a47769e95b5aed257fb6bfb946c1f2d419c"}
Jul 07 11:11:38.174310 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:38.174211    2570 scope.go:117] "RemoveContainer" containerID="7fa52da130bdfdf6ec86ca00321110e8e73ed9107562d8fb338e86a6275ffef4"
Jul 07 11:11:38.174566 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:38.174551    2570 scope.go:117] "RemoveContainer" containerID="a1299409496a2b71caab3591acf44a47769e95b5aed257fb6bfb946c1f2d419c"
Jul 07 11:11:38.174722 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:38.174704    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-4v7v8_openshift-console-operator(c8c17c90-3e07-4530-8411-d59cef9a1aff)\"" pod="openshift-console-operator/console-operator-68b757fb67-4v7v8" podUID="c8c17c90-3e07-4530-8411-d59cef9a1aff"
Jul 07 11:11:38.176730 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:38.176698    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-95lgm" event={"ID":"192f418c-f6e6-4775-876f-ef3d4f11b265","Type":"ContainerStarted","Data":"c04d291934a232bc520c0285edf2a4deccfb52854997e1b64d1c219db9f2b6e9"}
Jul 07 11:11:38.190964 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:38.190916    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-r6v22" podStartSLOduration=2.244581297 podStartE2EDuration="10.190903551s" totalImagesPullingTime="7.946322254s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:11:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:11:38.189970774 +0000 UTC m=+47.899867163" watchObservedRunningTime="2026-07-07 11:11:38.190903551 +0000 UTC m=+47.900799916"
Jul 07 11:11:38.215217 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:38.215169    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6h7pw" podStartSLOduration=2.481716192 podStartE2EDuration="10.215149231s" totalImagesPullingTime="7.733433039s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:11:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:11:38.213015088 +0000 UTC m=+47.922911476" watchObservedRunningTime="2026-07-07 11:11:38.215149231 +0000 UTC m=+47.925045619"
Jul 07 11:11:38.264365 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:38.264290    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-95lgm" podStartSLOduration=2.6664407629999998 podStartE2EDuration="10.264271652s" totalImagesPullingTime="7.597830889s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:11:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:11:38.263202768 +0000 UTC m=+47.973099156" watchObservedRunningTime="2026-07-07 11:11:38.264271652 +0000 UTC m=+47.974168043"
Jul 07 11:11:38.281978 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:38.281931    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-tt42z" podStartSLOduration=2.710482988 podStartE2EDuration="10.281914886s" totalImagesPullingTime="7.571431898s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:11:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:11:38.280436592 +0000 UTC m=+47.990332979" watchObservedRunningTime="2026-07-07 11:11:38.281914886 +0000 UTC m=+47.991811272"
Jul 07 11:11:38.368489 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:38.368460    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxztq_c20b78c4-cef9-4f47-ac6e-0ac4303eb857/konnectivity-agent/0.log" limit=1024
Jul 07 11:11:38.409107 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:38.409066    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cc937ff9-effb-4a24-a888-06fc5f4e5fe6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8kjb7\" (UID: \"cc937ff9-effb-4a24-a888-06fc5f4e5fe6\") " pod="openshift-insights/insights-runtime-extractor-8kjb7"
Jul 07 11:11:38.409307 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:38.409203    2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 11:11:38.409307 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:38.409208    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cc937ff9-effb-4a24-a888-06fc5f4e5fe6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8kjb7\" (UID: \"cc937ff9-effb-4a24-a888-06fc5f4e5fe6\") " pod="openshift-insights/insights-runtime-extractor-8kjb7"
Jul 07 11:11:38.409307 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:38.409297    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 11:11:38.409498 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:38.409300    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cc937ff9-effb-4a24-a888-06fc5f4e5fe6-kube-rbac-proxy-cm podName:cc937ff9-effb-4a24-a888-06fc5f4e5fe6 nodeName:}" failed. No retries permitted until 2026-07-07 11:11:39.409263882 +0000 UTC m=+49.119160261 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cc937ff9-effb-4a24-a888-06fc5f4e5fe6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8kjb7" (UID: "cc937ff9-effb-4a24-a888-06fc5f4e5fe6") : configmap "kube-rbac-proxy" not found
Jul 07 11:11:38.409498 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:38.409380    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc937ff9-effb-4a24-a888-06fc5f4e5fe6-insights-runtime-extractor-tls podName:cc937ff9-effb-4a24-a888-06fc5f4e5fe6 nodeName:}" failed. No retries permitted until 2026-07-07 11:11:39.409364016 +0000 UTC m=+49.119260385 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cc937ff9-effb-4a24-a888-06fc5f4e5fe6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8kjb7" (UID: "cc937ff9-effb-4a24-a888-06fc5f4e5fe6") : secret "insights-runtime-extractor-tls" not found
Jul 07 11:11:38.469828 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:38.469743    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxztq_c20b78c4-cef9-4f47-ac6e-0ac4303eb857/konnectivity-agent/0.log" limit=1024
Jul 07 11:11:38.579769 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:38.579720    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxztq_c20b78c4-cef9-4f47-ac6e-0ac4303eb857/konnectivity-agent/0.log" limit=1024
Jul 07 11:11:38.647011 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:38.646942    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxztq_c20b78c4-cef9-4f47-ac6e-0ac4303eb857/konnectivity-agent/0.log" limit=1024
Jul 07 11:11:38.708590 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:38.708556    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxztq_c20b78c4-cef9-4f47-ac6e-0ac4303eb857/konnectivity-agent/0.log" limit=1024
Jul 07 11:11:38.857408 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:38.857370    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-4v7v8"
Jul 07 11:11:38.857548 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:38.857422    2570 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-4v7v8"
Jul 07 11:11:39.055146 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:39.055119    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxztq_c20b78c4-cef9-4f47-ac6e-0ac4303eb857/konnectivity-agent/0.log" limit=1024
Jul 07 11:11:39.156657 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:39.156573    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxztq_c20b78c4-cef9-4f47-ac6e-0ac4303eb857/konnectivity-agent/0.log" limit=1024
Jul 07 11:11:39.183027 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:39.183000    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4v7v8_c8c17c90-3e07-4530-8411-d59cef9a1aff/console-operator/1.log"
Jul 07 11:11:39.183390 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:39.183375    2570 scope.go:117] "RemoveContainer" containerID="a1299409496a2b71caab3591acf44a47769e95b5aed257fb6bfb946c1f2d419c"
Jul 07 11:11:39.183561 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:39.183537    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-4v7v8_openshift-console-operator(c8c17c90-3e07-4530-8411-d59cef9a1aff)\"" pod="openshift-console-operator/console-operator-68b757fb67-4v7v8" podUID="c8c17c90-3e07-4530-8411-d59cef9a1aff"
Jul 07 11:11:39.419275 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:39.419192    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cc937ff9-effb-4a24-a888-06fc5f4e5fe6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8kjb7\" (UID: \"cc937ff9-effb-4a24-a888-06fc5f4e5fe6\") " pod="openshift-insights/insights-runtime-extractor-8kjb7"
Jul 07 11:11:39.419441 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:39.419290    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cc937ff9-effb-4a24-a888-06fc5f4e5fe6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8kjb7\" (UID: \"cc937ff9-effb-4a24-a888-06fc5f4e5fe6\") " pod="openshift-insights/insights-runtime-extractor-8kjb7"
Jul 07 11:11:39.419441 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:39.419316    2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 11:11:39.419441 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:39.419407    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cc937ff9-effb-4a24-a888-06fc5f4e5fe6-kube-rbac-proxy-cm podName:cc937ff9-effb-4a24-a888-06fc5f4e5fe6 nodeName:}" failed. No retries permitted until 2026-07-07 11:11:41.419389505 +0000 UTC m=+51.129285871 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cc937ff9-effb-4a24-a888-06fc5f4e5fe6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8kjb7" (UID: "cc937ff9-effb-4a24-a888-06fc5f4e5fe6") : configmap "kube-rbac-proxy" not found
Jul 07 11:11:39.419545 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:39.419447    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 11:11:39.419545 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:39.419506    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc937ff9-effb-4a24-a888-06fc5f4e5fe6-insights-runtime-extractor-tls podName:cc937ff9-effb-4a24-a888-06fc5f4e5fe6 nodeName:}" failed. No retries permitted until 2026-07-07 11:11:41.419489788 +0000 UTC m=+51.129386156 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cc937ff9-effb-4a24-a888-06fc5f4e5fe6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8kjb7" (UID: "cc937ff9-effb-4a24-a888-06fc5f4e5fe6") : secret "insights-runtime-extractor-tls" not found
Jul 07 11:11:39.443118 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:39.443089    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rqrgc_4ea618ab-a367-4d71-b0bd-e82e15e1f080/dns-node-resolver/0.log"
Jul 07 11:11:39.762969 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:39.762935    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxztq_c20b78c4-cef9-4f47-ac6e-0ac4303eb857/konnectivity-agent/0.log" limit=1024
Jul 07 11:11:39.866042 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:39.866009    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxztq_c20b78c4-cef9-4f47-ac6e-0ac4303eb857/konnectivity-agent/0.log" limit=1024
Jul 07 11:11:40.089902 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:40.089820    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxztq_c20b78c4-cef9-4f47-ac6e-0ac4303eb857/konnectivity-agent/0.log" limit=1024
Jul 07 11:11:40.187109 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:40.187081    2570 scope.go:117] "RemoveContainer" containerID="a1299409496a2b71caab3591acf44a47769e95b5aed257fb6bfb946c1f2d419c"
Jul 07 11:11:40.187262 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:40.187227    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-4v7v8_openshift-console-operator(c8c17c90-3e07-4530-8411-d59cef9a1aff)\"" pod="openshift-console-operator/console-operator-68b757fb67-4v7v8" podUID="c8c17c90-3e07-4530-8411-d59cef9a1aff"
Jul 07 11:11:40.188914 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:40.188890    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxztq_c20b78c4-cef9-4f47-ac6e-0ac4303eb857/konnectivity-agent/0.log" limit=1024
Jul 07 11:11:40.289693 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:40.289661    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dxztq_c20b78c4-cef9-4f47-ac6e-0ac4303eb857/konnectivity-agent/0.log" limit=1024
Jul 07 11:11:40.443561 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:40.443493    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-x8c7r_2827544c-22cd-4454-86a0-fc57d09ea878/node-ca/0.log"
Jul 07 11:11:41.042607 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:41.042577    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wlbnq_3e1e8d03-ef0d-49d9-a760-64039277242c/serve-healthcheck-canary/0.log"
Jul 07 11:11:41.436828 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:41.436739    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cc937ff9-effb-4a24-a888-06fc5f4e5fe6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8kjb7\" (UID: \"cc937ff9-effb-4a24-a888-06fc5f4e5fe6\") " pod="openshift-insights/insights-runtime-extractor-8kjb7"
Jul 07 11:11:41.436994 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:41.436831    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cc937ff9-effb-4a24-a888-06fc5f4e5fe6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8kjb7\" (UID: \"cc937ff9-effb-4a24-a888-06fc5f4e5fe6\") " pod="openshift-insights/insights-runtime-extractor-8kjb7"
Jul 07 11:11:41.436994 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:41.436893    2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 11:11:41.436994 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:41.436973    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cc937ff9-effb-4a24-a888-06fc5f4e5fe6-kube-rbac-proxy-cm podName:cc937ff9-effb-4a24-a888-06fc5f4e5fe6 nodeName:}" failed. No retries permitted until 2026-07-07 11:11:45.436951296 +0000 UTC m=+55.146847666 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cc937ff9-effb-4a24-a888-06fc5f4e5fe6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8kjb7" (UID: "cc937ff9-effb-4a24-a888-06fc5f4e5fe6") : configmap "kube-rbac-proxy" not found
Jul 07 11:11:41.437148 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:41.436995    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 11:11:41.437148 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:41.437048    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc937ff9-effb-4a24-a888-06fc5f4e5fe6-insights-runtime-extractor-tls podName:cc937ff9-effb-4a24-a888-06fc5f4e5fe6 nodeName:}" failed. No retries permitted until 2026-07-07 11:11:45.437033479 +0000 UTC m=+55.146929857 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cc937ff9-effb-4a24-a888-06fc5f4e5fe6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8kjb7" (UID: "cc937ff9-effb-4a24-a888-06fc5f4e5fe6") : secret "insights-runtime-extractor-tls" not found
Jul 07 11:11:42.041436 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:42.041398    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/daf8c8f2-2b41-4800-a056-1f1b1d9714b7-metrics-tls\") pod \"dns-default-64n4d\" (UID: \"daf8c8f2-2b41-4800-a056-1f1b1d9714b7\") " pod="openshift-dns/dns-default-64n4d"
Jul 07 11:11:42.041614 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:42.041448    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc075655-13c0-4026-9ed2-8c4cc2a71dab-registry-tls\") pod \"image-registry-9bb4d966c-zfwhd\" (UID: \"cc075655-13c0-4026-9ed2-8c4cc2a71dab\") " pod="openshift-image-registry/image-registry-9bb4d966c-zfwhd"
Jul 07 11:11:42.041614 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:42.041566    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 11:11:42.041614 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:42.041571    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 11:11:42.041711 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:42.041630    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/daf8c8f2-2b41-4800-a056-1f1b1d9714b7-metrics-tls podName:daf8c8f2-2b41-4800-a056-1f1b1d9714b7 nodeName:}" failed. No retries permitted until 2026-07-07 11:11:58.04161511 +0000 UTC m=+67.751511475 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/daf8c8f2-2b41-4800-a056-1f1b1d9714b7-metrics-tls") pod "dns-default-64n4d" (UID: "daf8c8f2-2b41-4800-a056-1f1b1d9714b7") : secret "dns-default-metrics-tls" not found
Jul 07 11:11:42.041711 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:42.041578    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9bb4d966c-zfwhd: secret "image-registry-tls" not found
Jul 07 11:11:42.041711 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:42.041665    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cc075655-13c0-4026-9ed2-8c4cc2a71dab-registry-tls podName:cc075655-13c0-4026-9ed2-8c4cc2a71dab nodeName:}" failed. No retries permitted until 2026-07-07 11:11:58.041659025 +0000 UTC m=+67.751555424 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cc075655-13c0-4026-9ed2-8c4cc2a71dab-registry-tls") pod "image-registry-9bb4d966c-zfwhd" (UID: "cc075655-13c0-4026-9ed2-8c4cc2a71dab") : secret "image-registry-tls" not found
Jul 07 11:11:44.362904 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:44.362863    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91203aa4-b720-4390-a2c0-5c553ffd79f8-service-ca-bundle\") pod \"router-default-5dc4cf6785-bflm8\" (UID: \"91203aa4-b720-4390-a2c0-5c553ffd79f8\") " pod="openshift-ingress/router-default-5dc4cf6785-bflm8"
Jul 07 11:11:44.363516 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:44.363033    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/91203aa4-b720-4390-a2c0-5c553ffd79f8-service-ca-bundle podName:91203aa4-b720-4390-a2c0-5c553ffd79f8 nodeName:}" failed. No retries permitted until 2026-07-07 11:12:00.36301696 +0000 UTC m=+70.072913324 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/91203aa4-b720-4390-a2c0-5c553ffd79f8-service-ca-bundle") pod "router-default-5dc4cf6785-bflm8" (UID: "91203aa4-b720-4390-a2c0-5c553ffd79f8") : configmap references non-existent config key: service-ca.crt
Jul 07 11:11:44.363516 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:44.363124    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91203aa4-b720-4390-a2c0-5c553ffd79f8-metrics-certs\") pod \"router-default-5dc4cf6785-bflm8\" (UID: \"91203aa4-b720-4390-a2c0-5c553ffd79f8\") " pod="openshift-ingress/router-default-5dc4cf6785-bflm8"
Jul 07 11:11:44.363516 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:44.363229    2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 07 11:11:44.363516 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:44.363264    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91203aa4-b720-4390-a2c0-5c553ffd79f8-metrics-certs podName:91203aa4-b720-4390-a2c0-5c553ffd79f8 nodeName:}" failed. No retries permitted until 2026-07-07 11:12:00.363255767 +0000 UTC m=+70.073152132 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/91203aa4-b720-4390-a2c0-5c553ffd79f8-metrics-certs") pod "router-default-5dc4cf6785-bflm8" (UID: "91203aa4-b720-4390-a2c0-5c553ffd79f8") : secret "router-metrics-certs-default" not found
Jul 07 11:11:44.464176 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:44.464136    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/df6d16a6-2593-4477-ae7a-eca848817f34-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-9pdtd\" (UID: \"df6d16a6-2593-4477-ae7a-eca848817f34\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-9pdtd"
Jul 07 11:11:44.464369 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:44.464296    2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 07 11:11:44.464415 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:44.464387    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df6d16a6-2593-4477-ae7a-eca848817f34-cluster-monitoring-operator-tls podName:df6d16a6-2593-4477-ae7a-eca848817f34 nodeName:}" failed. No retries permitted until 2026-07-07 11:12:00.464368739 +0000 UTC m=+70.174265103 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/df6d16a6-2593-4477-ae7a-eca848817f34-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-9pdtd" (UID: "df6d16a6-2593-4477-ae7a-eca848817f34") : secret "cluster-monitoring-operator-tls" not found
Jul 07 11:11:44.565124 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:44.565087    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f9ec8255-2877-4c6a-b248-b50d64978f7a-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-vj4hq\" (UID: \"f9ec8255-2877-4c6a-b248-b50d64978f7a\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-vj4hq"
Jul 07 11:11:44.565279 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:44.565158    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/63454a74-dfe7-497f-9f3d-0725bc111250-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-lmnhq\" (UID: \"63454a74-dfe7-497f-9f3d-0725bc111250\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-lmnhq"
Jul 07 11:11:44.565279 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:44.565238    2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 07 11:11:44.565279 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:44.565244    2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 07 11:11:44.565410 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:44.565302    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9ec8255-2877-4c6a-b248-b50d64978f7a-networking-console-plugin-cert podName:f9ec8255-2877-4c6a-b248-b50d64978f7a nodeName:}" failed. No retries permitted until 2026-07-07 11:12:00.565287744 +0000 UTC m=+70.275184108 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f9ec8255-2877-4c6a-b248-b50d64978f7a-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-vj4hq" (UID: "f9ec8255-2877-4c6a-b248-b50d64978f7a") : secret "networking-console-plugin-cert" not found
Jul 07 11:11:44.565410 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:44.565345    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63454a74-dfe7-497f-9f3d-0725bc111250-samples-operator-tls podName:63454a74-dfe7-497f-9f3d-0725bc111250 nodeName:}" failed. No retries permitted until 2026-07-07 11:12:00.56531341 +0000 UTC m=+70.275209775 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/63454a74-dfe7-497f-9f3d-0725bc111250-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-lmnhq" (UID: "63454a74-dfe7-497f-9f3d-0725bc111250") : secret "samples-operator-tls" not found
Jul 07 11:11:45.474562 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:45.474510    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cc937ff9-effb-4a24-a888-06fc5f4e5fe6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8kjb7\" (UID: \"cc937ff9-effb-4a24-a888-06fc5f4e5fe6\") " pod="openshift-insights/insights-runtime-extractor-8kjb7"
Jul 07 11:11:45.475009 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:45.474635    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cc937ff9-effb-4a24-a888-06fc5f4e5fe6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8kjb7\" (UID: \"cc937ff9-effb-4a24-a888-06fc5f4e5fe6\") " pod="openshift-insights/insights-runtime-extractor-8kjb7"
Jul 07 11:11:45.475009 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:45.474666    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 11:11:45.475009 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:45.474733    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc937ff9-effb-4a24-a888-06fc5f4e5fe6-insights-runtime-extractor-tls podName:cc937ff9-effb-4a24-a888-06fc5f4e5fe6 nodeName:}" failed. No retries permitted until 2026-07-07 11:11:53.474718431 +0000 UTC m=+63.184614798 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cc937ff9-effb-4a24-a888-06fc5f4e5fe6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8kjb7" (UID: "cc937ff9-effb-4a24-a888-06fc5f4e5fe6") : secret "insights-runtime-extractor-tls" not found
Jul 07 11:11:45.475138 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:45.475106    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cc937ff9-effb-4a24-a888-06fc5f4e5fe6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8kjb7\" (UID: \"cc937ff9-effb-4a24-a888-06fc5f4e5fe6\") " pod="openshift-insights/insights-runtime-extractor-8kjb7"
Jul 07 11:11:52.078394 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:52.078359    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dbsg9"
Jul 07 11:11:52.815048 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:52.815010    2570 scope.go:117] "RemoveContainer" containerID="a1299409496a2b71caab3591acf44a47769e95b5aed257fb6bfb946c1f2d419c"
Jul 07 11:11:53.226748 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:53.226674    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4v7v8_c8c17c90-3e07-4530-8411-d59cef9a1aff/console-operator/2.log"
Jul 07 11:11:53.227131 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:53.227114    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4v7v8_c8c17c90-3e07-4530-8411-d59cef9a1aff/console-operator/1.log"
Jul 07 11:11:53.227172 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:53.227149    2570 generic.go:356] "Generic (PLEG): container finished" podID="c8c17c90-3e07-4530-8411-d59cef9a1aff" containerID="248c742829f41d119f62e9f783a582c6e1b35a6810ab916cc2a3cdc51ee2c9a4" exitCode=255
Jul 07 11:11:53.227209 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:53.227179    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-4v7v8" event={"ID":"c8c17c90-3e07-4530-8411-d59cef9a1aff","Type":"ContainerDied","Data":"248c742829f41d119f62e9f783a582c6e1b35a6810ab916cc2a3cdc51ee2c9a4"}
Jul 07 11:11:53.227245 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:53.227220    2570 scope.go:117] "RemoveContainer" containerID="a1299409496a2b71caab3591acf44a47769e95b5aed257fb6bfb946c1f2d419c"
Jul 07 11:11:53.227621 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:53.227598    2570 scope.go:117] "RemoveContainer" containerID="248c742829f41d119f62e9f783a582c6e1b35a6810ab916cc2a3cdc51ee2c9a4"
Jul 07 11:11:53.227829 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:53.227803    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-4v7v8_openshift-console-operator(c8c17c90-3e07-4530-8411-d59cef9a1aff)\"" pod="openshift-console-operator/console-operator-68b757fb67-4v7v8" podUID="c8c17c90-3e07-4530-8411-d59cef9a1aff"
Jul 07 11:11:53.557146 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:53.557091    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cc937ff9-effb-4a24-a888-06fc5f4e5fe6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8kjb7\" (UID: \"cc937ff9-effb-4a24-a888-06fc5f4e5fe6\") " pod="openshift-insights/insights-runtime-extractor-8kjb7"
Jul 07 11:11:53.559542 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:53.559521    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cc937ff9-effb-4a24-a888-06fc5f4e5fe6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8kjb7\" (UID: \"cc937ff9-effb-4a24-a888-06fc5f4e5fe6\") " pod="openshift-insights/insights-runtime-extractor-8kjb7"
Jul 07 11:11:53.601262 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:53.601227    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tnjrv\""
Jul 07 11:11:53.608568 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:53.608548    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8kjb7"
Jul 07 11:11:53.742677 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:53.742652    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8kjb7"]
Jul 07 11:11:53.745001 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:11:53.744973    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc937ff9_effb_4a24_a888_06fc5f4e5fe6.slice/crio-f68b0281856112ab686f7304ac32c22af7e72c57833c287fa16a30a22966766d WatchSource:0}: Error finding container f68b0281856112ab686f7304ac32c22af7e72c57833c287fa16a30a22966766d: Status 404 returned error can't find the container with id f68b0281856112ab686f7304ac32c22af7e72c57833c287fa16a30a22966766d
Jul 07 11:11:54.234764 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:54.234734    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4v7v8_c8c17c90-3e07-4530-8411-d59cef9a1aff/console-operator/2.log"
Jul 07 11:11:54.236624 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:54.236531    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8kjb7" event={"ID":"cc937ff9-effb-4a24-a888-06fc5f4e5fe6","Type":"ContainerStarted","Data":"f68b0281856112ab686f7304ac32c22af7e72c57833c287fa16a30a22966766d"}
Jul 07 11:11:56.485524 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:56.485482    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/38a7b9ab-6691-41f8-a3b7-e47cec808b30-metrics-certs\") pod \"network-metrics-daemon-m7hkl\" (UID: \"38a7b9ab-6691-41f8-a3b7-e47cec808b30\") " pod="openshift-multus/network-metrics-daemon-m7hkl"
Jul 07 11:11:56.488830 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:56.488810    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 07 11:11:56.498550 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:56.498522    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/38a7b9ab-6691-41f8-a3b7-e47cec808b30-metrics-certs\") pod \"network-metrics-daemon-m7hkl\" (UID: \"38a7b9ab-6691-41f8-a3b7-e47cec808b30\") " pod="openshift-multus/network-metrics-daemon-m7hkl"
Jul 07 11:11:56.537885 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:56.537856    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7b27w\""
Jul 07 11:11:56.545784 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:56.545757    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m7hkl"
Jul 07 11:11:56.587172 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:56.587139    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a89cad11-0010-4ab5-bc82-04a3cd7a3e7b-original-pull-secret\") pod \"global-pull-secret-syncer-px74n\" (UID: \"a89cad11-0010-4ab5-bc82-04a3cd7a3e7b\") " pod="kube-system/global-pull-secret-syncer-px74n"
Jul 07 11:11:56.593762 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:56.593727    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 07 11:11:56.600255 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:56.600228    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a89cad11-0010-4ab5-bc82-04a3cd7a3e7b-original-pull-secret\") pod \"global-pull-secret-syncer-px74n\" (UID: \"a89cad11-0010-4ab5-bc82-04a3cd7a3e7b\") " pod="kube-system/global-pull-secret-syncer-px74n"
Jul 07 11:11:56.688692 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:56.688652    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kbzb\" (UniqueName: \"kubernetes.io/projected/f971d15b-e001-45d9-ba6f-e0b60efc55cc-kube-api-access-4kbzb\") pod \"network-check-target-n5mhp\" (UID: \"f971d15b-e001-45d9-ba6f-e0b60efc55cc\") " pod="openshift-network-diagnostics/network-check-target-n5mhp"
Jul 07 11:11:56.691660 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:56.691622    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kbzb\" (UniqueName: \"kubernetes.io/projected/f971d15b-e001-45d9-ba6f-e0b60efc55cc-kube-api-access-4kbzb\") pod \"network-check-target-n5mhp\" (UID: \"f971d15b-e001-45d9-ba6f-e0b60efc55cc\") " pod="openshift-network-diagnostics/network-check-target-n5mhp"
Jul 07 11:11:56.831638 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:56.831563    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cn2cx\""
Jul 07 11:11:56.839432 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:56.839401    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n5mhp"
Jul 07 11:11:56.841759 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:56.841736    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-px74n"
Jul 07 11:11:57.576218 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:57.576190    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-px74n"]
Jul 07 11:11:57.577801 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:11:57.577772    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda89cad11_0010_4ab5_bc82_04a3cd7a3e7b.slice/crio-5354151eaf0478ba21b331779524257480001b399594faa1b1d472ab26b10c56 WatchSource:0}: Error finding container 5354151eaf0478ba21b331779524257480001b399594faa1b1d472ab26b10c56: Status 404 returned error can't find the container with id 5354151eaf0478ba21b331779524257480001b399594faa1b1d472ab26b10c56
Jul 07 11:11:57.594092 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:57.594064    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-m7hkl"]
Jul 07 11:11:57.598143 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:11:57.598116    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38a7b9ab_6691_41f8_a3b7_e47cec808b30.slice/crio-4d084dd1891632e48fc1c83fcffcfc1a6c2eab8a3940e5600f549c54214f6da2 WatchSource:0}: Error finding container 4d084dd1891632e48fc1c83fcffcfc1a6c2eab8a3940e5600f549c54214f6da2: Status 404 returned error can't find the container with id 4d084dd1891632e48fc1c83fcffcfc1a6c2eab8a3940e5600f549c54214f6da2
Jul 07 11:11:57.609813 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:57.609790    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-n5mhp"]
Jul 07 11:11:57.611936 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:11:57.611913    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf971d15b_e001_45d9_ba6f_e0b60efc55cc.slice/crio-e605bc8a1b5bb3fd982f511de31f49ce405c7b3ed366e0e22cc3c670853ac56d WatchSource:0}: Error finding container e605bc8a1b5bb3fd982f511de31f49ce405c7b3ed366e0e22cc3c670853ac56d: Status 404 returned error can't find the container with id e605bc8a1b5bb3fd982f511de31f49ce405c7b3ed366e0e22cc3c670853ac56d
Jul 07 11:11:58.101261 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:58.101222    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/daf8c8f2-2b41-4800-a056-1f1b1d9714b7-metrics-tls\") pod \"dns-default-64n4d\" (UID: \"daf8c8f2-2b41-4800-a056-1f1b1d9714b7\") " pod="openshift-dns/dns-default-64n4d"
Jul 07 11:11:58.101458 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:58.101280    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc075655-13c0-4026-9ed2-8c4cc2a71dab-registry-tls\") pod \"image-registry-9bb4d966c-zfwhd\" (UID: \"cc075655-13c0-4026-9ed2-8c4cc2a71dab\") " pod="openshift-image-registry/image-registry-9bb4d966c-zfwhd"
Jul 07 11:11:58.103813 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:58.103773    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/daf8c8f2-2b41-4800-a056-1f1b1d9714b7-metrics-tls\") pod \"dns-default-64n4d\" (UID: \"daf8c8f2-2b41-4800-a056-1f1b1d9714b7\") " pod="openshift-dns/dns-default-64n4d"
Jul 07 11:11:58.103915 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:58.103827    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc075655-13c0-4026-9ed2-8c4cc2a71dab-registry-tls\") pod \"image-registry-9bb4d966c-zfwhd\" (UID: \"cc075655-13c0-4026-9ed2-8c4cc2a71dab\") " pod="openshift-image-registry/image-registry-9bb4d966c-zfwhd"
Jul 07 11:11:58.219238 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:58.219210    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-hg565\""
Jul 07 11:11:58.225945 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:58.225924    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9bb4d966c-zfwhd"
Jul 07 11:11:58.240171 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:58.239971    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2gjl2\""
Jul 07 11:11:58.246761 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:58.246702    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-64n4d"
Jul 07 11:11:58.254129 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:58.254047    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-px74n" event={"ID":"a89cad11-0010-4ab5-bc82-04a3cd7a3e7b","Type":"ContainerStarted","Data":"5354151eaf0478ba21b331779524257480001b399594faa1b1d472ab26b10c56"}
Jul 07 11:11:58.255674 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:58.255645    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-m7hkl" event={"ID":"38a7b9ab-6691-41f8-a3b7-e47cec808b30","Type":"ContainerStarted","Data":"4d084dd1891632e48fc1c83fcffcfc1a6c2eab8a3940e5600f549c54214f6da2"}
Jul 07 11:11:58.257758 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:58.257687    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-n5mhp" event={"ID":"f971d15b-e001-45d9-ba6f-e0b60efc55cc","Type":"ContainerStarted","Data":"595a2c093fd9a54fdbf7c40ad937a39e8f6c2edf49088b125760efc88582beb6"}
Jul 07 11:11:58.257758 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:58.257724    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-n5mhp" event={"ID":"f971d15b-e001-45d9-ba6f-e0b60efc55cc","Type":"ContainerStarted","Data":"e605bc8a1b5bb3fd982f511de31f49ce405c7b3ed366e0e22cc3c670853ac56d"}
Jul 07 11:11:58.257898 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:58.257821    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-n5mhp"
Jul 07 11:11:58.259619 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:58.259592    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8kjb7" event={"ID":"cc937ff9-effb-4a24-a888-06fc5f4e5fe6","Type":"ContainerStarted","Data":"7aaa56743e5f1de50f0ac87cf23ea5f0a803eb380aabdfc270ad9c91984dd6c3"}
Jul 07 11:11:58.284936 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:58.284471    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-n5mhp" podStartSLOduration=67.284449536 podStartE2EDuration="1m7.284449536s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:10:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:11:58.283848594 +0000 UTC m=+67.993744981" watchObservedRunningTime="2026-07-07 11:11:58.284449536 +0000 UTC m=+67.994345929"
Jul 07 11:11:58.441735 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:58.441704    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-9bb4d966c-zfwhd"]
Jul 07 11:11:58.446118 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:11:58.446086    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc075655_13c0_4026_9ed2_8c4cc2a71dab.slice/crio-1af2ace849a357649917c2fe26f54cf1f71e4e755ce69a9baa404bc076a52677 WatchSource:0}: Error finding container 1af2ace849a357649917c2fe26f54cf1f71e4e755ce69a9baa404bc076a52677: Status 404 returned error can't find the container with id 1af2ace849a357649917c2fe26f54cf1f71e4e755ce69a9baa404bc076a52677
Jul 07 11:11:58.450611 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:58.450579    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-64n4d"]
Jul 07 11:11:58.453781 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:11:58.453758    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaf8c8f2_2b41_4800_a056_1f1b1d9714b7.slice/crio-0fadd7a4c4d12e85bfe7a00084cd6ee643c539894a3b6abb03a7c52536577bb6 WatchSource:0}: Error finding container 0fadd7a4c4d12e85bfe7a00084cd6ee643c539894a3b6abb03a7c52536577bb6: Status 404 returned error can't find the container with id 0fadd7a4c4d12e85bfe7a00084cd6ee643c539894a3b6abb03a7c52536577bb6
Jul 07 11:11:58.857645 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:58.857605    2570 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-4v7v8"
Jul 07 11:11:58.858032 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:58.857683    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-4v7v8"
Jul 07 11:11:58.858074 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:58.858050    2570 scope.go:117] "RemoveContainer" containerID="248c742829f41d119f62e9f783a582c6e1b35a6810ab916cc2a3cdc51ee2c9a4"
Jul 07 11:11:58.858277 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:11:58.858251    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-4v7v8_openshift-console-operator(c8c17c90-3e07-4530-8411-d59cef9a1aff)\"" pod="openshift-console-operator/console-operator-68b757fb67-4v7v8" podUID="c8c17c90-3e07-4530-8411-d59cef9a1aff"
Jul 07 11:11:59.265537 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:59.265495    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-64n4d" event={"ID":"daf8c8f2-2b41-4800-a056-1f1b1d9714b7","Type":"ContainerStarted","Data":"0fadd7a4c4d12e85bfe7a00084cd6ee643c539894a3b6abb03a7c52536577bb6"}
Jul 07 11:11:59.267504 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:59.267474    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8kjb7" event={"ID":"cc937ff9-effb-4a24-a888-06fc5f4e5fe6","Type":"ContainerStarted","Data":"4e158c20d9c924eb23f0dad94eb323b0d9a1a428e50615b263557e5dc72b848f"}
Jul 07 11:11:59.269170 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:59.269143    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9bb4d966c-zfwhd" event={"ID":"cc075655-13c0-4026-9ed2-8c4cc2a71dab","Type":"ContainerStarted","Data":"2811c01af62ae5f83b021a5c84cdcd050f3d6d19bc3ab55395cb8acf59c0c269"}
Jul 07 11:11:59.269263 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:59.269190    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9bb4d966c-zfwhd" event={"ID":"cc075655-13c0-4026-9ed2-8c4cc2a71dab","Type":"ContainerStarted","Data":"1af2ace849a357649917c2fe26f54cf1f71e4e755ce69a9baa404bc076a52677"}
Jul 07 11:11:59.269367 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:59.269259    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-9bb4d966c-zfwhd"
Jul 07 11:11:59.301702 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:11:59.301650    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-9bb4d966c-zfwhd" podStartSLOduration=45.301633664 podStartE2EDuration="45.301633664s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:11:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:11:59.299993625 +0000 UTC m=+69.009890012" watchObservedRunningTime="2026-07-07 11:11:59.301633664 +0000 UTC m=+69.011530051"
Jul 07 11:12:00.276111 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:00.276074    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-m7hkl" event={"ID":"38a7b9ab-6691-41f8-a3b7-e47cec808b30","Type":"ContainerStarted","Data":"88b25f9411b515cbf84c646d6c4026fc27269b00e7263cdbf2908d4f6f480406"}
Jul 07 11:12:00.276643 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:00.276138    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-m7hkl" event={"ID":"38a7b9ab-6691-41f8-a3b7-e47cec808b30","Type":"ContainerStarted","Data":"c32a9b3283951782d05cefc580e78d2cce81839a09c9b1a7527bc943a96787ba"}
Jul 07 11:12:00.297756 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:00.296394    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-m7hkl" podStartSLOduration=67.614170747 podStartE2EDuration="1m9.296373058s" totalImagesPullingTime="1.682202311s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:10:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:12:00.296139589 +0000 UTC m=+70.006035979" watchObservedRunningTime="2026-07-07 11:12:00.296373058 +0000 UTC m=+70.006269460"
Jul 07 11:12:00.426510 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:00.426475    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91203aa4-b720-4390-a2c0-5c553ffd79f8-metrics-certs\") pod \"router-default-5dc4cf6785-bflm8\" (UID: \"91203aa4-b720-4390-a2c0-5c553ffd79f8\") " pod="openshift-ingress/router-default-5dc4cf6785-bflm8"
Jul 07 11:12:00.426669 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:00.426553    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91203aa4-b720-4390-a2c0-5c553ffd79f8-service-ca-bundle\") pod \"router-default-5dc4cf6785-bflm8\" (UID: \"91203aa4-b720-4390-a2c0-5c553ffd79f8\") " pod="openshift-ingress/router-default-5dc4cf6785-bflm8"
Jul 07 11:12:00.427155 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:00.427132    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91203aa4-b720-4390-a2c0-5c553ffd79f8-service-ca-bundle\") pod \"router-default-5dc4cf6785-bflm8\" (UID: \"91203aa4-b720-4390-a2c0-5c553ffd79f8\") " pod="openshift-ingress/router-default-5dc4cf6785-bflm8"
Jul 07 11:12:00.430206 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:00.430182    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91203aa4-b720-4390-a2c0-5c553ffd79f8-metrics-certs\") pod \"router-default-5dc4cf6785-bflm8\" (UID: \"91203aa4-b720-4390-a2c0-5c553ffd79f8\") " pod="openshift-ingress/router-default-5dc4cf6785-bflm8"
Jul 07 11:12:00.527846 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:00.527752    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/df6d16a6-2593-4477-ae7a-eca848817f34-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-9pdtd\" (UID: \"df6d16a6-2593-4477-ae7a-eca848817f34\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-9pdtd"
Jul 07 11:12:00.530505 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:00.530477    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/df6d16a6-2593-4477-ae7a-eca848817f34-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-9pdtd\" (UID: \"df6d16a6-2593-4477-ae7a-eca848817f34\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-9pdtd"
Jul 07 11:12:00.628740 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:00.628696    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/63454a74-dfe7-497f-9f3d-0725bc111250-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-lmnhq\" (UID: \"63454a74-dfe7-497f-9f3d-0725bc111250\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-lmnhq"
Jul 07 11:12:00.628924 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:00.628793    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f9ec8255-2877-4c6a-b248-b50d64978f7a-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-vj4hq\" (UID: \"f9ec8255-2877-4c6a-b248-b50d64978f7a\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-vj4hq"
Jul 07 11:12:00.634973 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:00.631819    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f9ec8255-2877-4c6a-b248-b50d64978f7a-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-vj4hq\" (UID: \"f9ec8255-2877-4c6a-b248-b50d64978f7a\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-vj4hq"
Jul 07 11:12:00.635767 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:00.635738    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/63454a74-dfe7-497f-9f3d-0725bc111250-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-lmnhq\" (UID: \"63454a74-dfe7-497f-9f3d-0725bc111250\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-lmnhq"
Jul 07 11:12:00.676530 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:00.676493    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-zdx2d\""
Jul 07 11:12:00.684276 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:00.684246    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5dc4cf6785-bflm8"
Jul 07 11:12:00.735028 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:00.734995    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-chkxh\""
Jul 07 11:12:00.742571 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:00.742534    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-9pdtd"
Jul 07 11:12:00.763557 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:00.763531    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-5lm6j\""
Jul 07 11:12:00.771374 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:00.771313    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-vj4hq"
Jul 07 11:12:00.790346 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:00.790256    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-jr4xl\""
Jul 07 11:12:00.797629 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:00.797606    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-lmnhq"
Jul 07 11:12:03.654501 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:03.653050    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-vj4hq"]
Jul 07 11:12:03.689975 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:03.689910    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-lmnhq"]
Jul 07 11:12:03.895244 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:03.895221    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-9pdtd"]
Jul 07 11:12:03.897597 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:12:03.897570    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf6d16a6_2593_4477_ae7a_eca848817f34.slice/crio-660c9e1435cf6ddca5b103c6a1bc20eb8326666afb605c8c12230698c5c8f49b WatchSource:0}: Error finding container 660c9e1435cf6ddca5b103c6a1bc20eb8326666afb605c8c12230698c5c8f49b: Status 404 returned error can't find the container with id 660c9e1435cf6ddca5b103c6a1bc20eb8326666afb605c8c12230698c5c8f49b
Jul 07 11:12:03.898401 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:03.898379    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5dc4cf6785-bflm8"]
Jul 07 11:12:03.900069 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:12:03.900046    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91203aa4_b720_4390_a2c0_5c553ffd79f8.slice/crio-be7c9beadcc868f02f99eb22f48952e657f88d2e143fd0bf4d95dc0f74cc16f6 WatchSource:0}: Error finding container be7c9beadcc868f02f99eb22f48952e657f88d2e143fd0bf4d95dc0f74cc16f6: Status 404 returned error can't find the container with id be7c9beadcc868f02f99eb22f48952e657f88d2e143fd0bf4d95dc0f74cc16f6
Jul 07 11:12:03.952669 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:03.952650    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79dc7c57c7-xgtwp"]
Jul 07 11:12:03.959740 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:03.959714    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79dc7c57c7-xgtwp"
Jul 07 11:12:03.965776 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:03.965749    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 07 11:12:03.966355 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:03.965750    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 07 11:12:03.966355 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:03.965797    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 07 11:12:03.966355 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:03.965838    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 07 11:12:03.966932 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:03.966891    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 07 11:12:03.967141 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:03.967124    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 07 11:12:03.967466 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:03.967357    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 07 11:12:03.971852 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:03.971831    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79dc7c57c7-xgtwp"]
Jul 07 11:12:04.057682 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:04.057652    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2e8f3fdb-cc2c-4305-a9e3-c2b7ec951dd7-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-79dc7c57c7-xgtwp\" (UID: \"2e8f3fdb-cc2c-4305-a9e3-c2b7ec951dd7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79dc7c57c7-xgtwp"
Jul 07 11:12:04.057682 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:04.057682    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rmsd\" (UniqueName: \"kubernetes.io/projected/2e8f3fdb-cc2c-4305-a9e3-c2b7ec951dd7-kube-api-access-5rmsd\") pod \"cluster-proxy-proxy-agent-79dc7c57c7-xgtwp\" (UID: \"2e8f3fdb-cc2c-4305-a9e3-c2b7ec951dd7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79dc7c57c7-xgtwp"
Jul 07 11:12:04.057879 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:04.057706    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/2e8f3fdb-cc2c-4305-a9e3-c2b7ec951dd7-ca\") pod \"cluster-proxy-proxy-agent-79dc7c57c7-xgtwp\" (UID: \"2e8f3fdb-cc2c-4305-a9e3-c2b7ec951dd7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79dc7c57c7-xgtwp"
Jul 07 11:12:04.057879 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:04.057788    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/2e8f3fdb-cc2c-4305-a9e3-c2b7ec951dd7-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-79dc7c57c7-xgtwp\" (UID: \"2e8f3fdb-cc2c-4305-a9e3-c2b7ec951dd7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79dc7c57c7-xgtwp"
Jul 07 11:12:04.057879 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:04.057804    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/2e8f3fdb-cc2c-4305-a9e3-c2b7ec951dd7-hub\") pod \"cluster-proxy-proxy-agent-79dc7c57c7-xgtwp\" (UID: \"2e8f3fdb-cc2c-4305-a9e3-c2b7ec951dd7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79dc7c57c7-xgtwp"
Jul 07 11:12:04.057879 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:04.057822    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e8f3fdb-cc2c-4305-a9e3-c2b7ec951dd7-additional-service-ca\") pod \"cluster-proxy-proxy-agent-79dc7c57c7-xgtwp\" (UID: \"2e8f3fdb-cc2c-4305-a9e3-c2b7ec951dd7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79dc7c57c7-xgtwp"
Jul 07 11:12:04.158703 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:04.158608    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/2e8f3fdb-cc2c-4305-a9e3-c2b7ec951dd7-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-79dc7c57c7-xgtwp\" (UID: \"2e8f3fdb-cc2c-4305-a9e3-c2b7ec951dd7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79dc7c57c7-xgtwp"
Jul 07 11:12:04.158703 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:04.158651    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/2e8f3fdb-cc2c-4305-a9e3-c2b7ec951dd7-hub\") pod \"cluster-proxy-proxy-agent-79dc7c57c7-xgtwp\" (UID: \"2e8f3fdb-cc2c-4305-a9e3-c2b7ec951dd7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79dc7c57c7-xgtwp"
Jul 07 11:12:04.158703 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:04.158673    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e8f3fdb-cc2c-4305-a9e3-c2b7ec951dd7-additional-service-ca\") pod \"cluster-proxy-proxy-agent-79dc7c57c7-xgtwp\" (UID: \"2e8f3fdb-cc2c-4305-a9e3-c2b7ec951dd7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79dc7c57c7-xgtwp"
Jul 07 11:12:04.158703 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:04.158724    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2e8f3fdb-cc2c-4305-a9e3-c2b7ec951dd7-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-79dc7c57c7-xgtwp\" (UID: \"2e8f3fdb-cc2c-4305-a9e3-c2b7ec951dd7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79dc7c57c7-xgtwp"
Jul 07 11:12:04.159018 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:04.158749    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rmsd\" (UniqueName: \"kubernetes.io/projected/2e8f3fdb-cc2c-4305-a9e3-c2b7ec951dd7-kube-api-access-5rmsd\") pod \"cluster-proxy-proxy-agent-79dc7c57c7-xgtwp\" (UID: \"2e8f3fdb-cc2c-4305-a9e3-c2b7ec951dd7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79dc7c57c7-xgtwp"
Jul 07 11:12:04.159018 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:04.158786    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/2e8f3fdb-cc2c-4305-a9e3-c2b7ec951dd7-ca\") pod \"cluster-proxy-proxy-agent-79dc7c57c7-xgtwp\" (UID: \"2e8f3fdb-cc2c-4305-a9e3-c2b7ec951dd7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79dc7c57c7-xgtwp"
Jul 07 11:12:04.159766 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:04.159738    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e8f3fdb-cc2c-4305-a9e3-c2b7ec951dd7-additional-service-ca\") pod \"cluster-proxy-proxy-agent-79dc7c57c7-xgtwp\" (UID: \"2e8f3fdb-cc2c-4305-a9e3-c2b7ec951dd7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79dc7c57c7-xgtwp"
Jul 07 11:12:04.161967 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:04.161945    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/2e8f3fdb-cc2c-4305-a9e3-c2b7ec951dd7-hub\") pod \"cluster-proxy-proxy-agent-79dc7c57c7-xgtwp\" (UID: \"2e8f3fdb-cc2c-4305-a9e3-c2b7ec951dd7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79dc7c57c7-xgtwp"
Jul 07 11:12:04.162095 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:04.161977    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/2e8f3fdb-cc2c-4305-a9e3-c2b7ec951dd7-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-79dc7c57c7-xgtwp\" (UID: \"2e8f3fdb-cc2c-4305-a9e3-c2b7ec951dd7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79dc7c57c7-xgtwp"
Jul 07 11:12:04.162298 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:04.162277    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/2e8f3fdb-cc2c-4305-a9e3-c2b7ec951dd7-ca\") pod \"cluster-proxy-proxy-agent-79dc7c57c7-xgtwp\" (UID: \"2e8f3fdb-cc2c-4305-a9e3-c2b7ec951dd7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79dc7c57c7-xgtwp"
Jul 07 11:12:04.162508 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:04.162488    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2e8f3fdb-cc2c-4305-a9e3-c2b7ec951dd7-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-79dc7c57c7-xgtwp\" (UID: \"2e8f3fdb-cc2c-4305-a9e3-c2b7ec951dd7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79dc7c57c7-xgtwp"
Jul 07 11:12:04.170138 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:04.170113    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rmsd\" (UniqueName: \"kubernetes.io/projected/2e8f3fdb-cc2c-4305-a9e3-c2b7ec951dd7-kube-api-access-5rmsd\") pod \"cluster-proxy-proxy-agent-79dc7c57c7-xgtwp\" (UID: \"2e8f3fdb-cc2c-4305-a9e3-c2b7ec951dd7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79dc7c57c7-xgtwp"
Jul 07 11:12:04.281359 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:04.281222    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79dc7c57c7-xgtwp"
Jul 07 11:12:04.294296 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:04.294255    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-px74n" event={"ID":"a89cad11-0010-4ab5-bc82-04a3cd7a3e7b","Type":"ContainerStarted","Data":"5d8dacd8cf022ac309bdfc5eb86a22ef37a3af483d99535074915434b54d67c7"}
Jul 07 11:12:04.297270 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:04.297227    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-64n4d" event={"ID":"daf8c8f2-2b41-4800-a056-1f1b1d9714b7","Type":"ContainerStarted","Data":"51d3a90ebf7b1d32ac7ce1ee82c07346f97ab9e952929a00492f0d7d3ad7b91d"}
Jul 07 11:12:04.297438 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:04.297296    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-64n4d" event={"ID":"daf8c8f2-2b41-4800-a056-1f1b1d9714b7","Type":"ContainerStarted","Data":"96b517c06557e09cf30788b3d0e0c1af14db9de5f29c7b7262e4513e1044b3cf"}
Jul 07 11:12:04.297483 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:04.297440    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-64n4d"
Jul 07 11:12:04.299767 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:04.299737    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-vj4hq" event={"ID":"f9ec8255-2877-4c6a-b248-b50d64978f7a","Type":"ContainerStarted","Data":"18ec48dee0f575bb6d935d87941abb045432fc451ba35b277b66ad20d58a08ab"}
Jul 07 11:12:04.302242 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:04.302211    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-lmnhq" event={"ID":"63454a74-dfe7-497f-9f3d-0725bc111250","Type":"ContainerStarted","Data":"3ecf4a914b708c6e938837f874ada9253a24b683f80e8283bbba212bbbe0a8e9"}
Jul 07 11:12:04.304374 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:04.304345    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5dc4cf6785-bflm8" event={"ID":"91203aa4-b720-4390-a2c0-5c553ffd79f8","Type":"ContainerStarted","Data":"8da5acb972fcc942e2e61875598e91fbebc7da3541b144d0bba16dc457e1331b"}
Jul 07 11:12:04.304480 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:04.304393    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5dc4cf6785-bflm8" event={"ID":"91203aa4-b720-4390-a2c0-5c553ffd79f8","Type":"ContainerStarted","Data":"be7c9beadcc868f02f99eb22f48952e657f88d2e143fd0bf4d95dc0f74cc16f6"}
Jul 07 11:12:04.308769 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:04.308742    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8kjb7" event={"ID":"cc937ff9-effb-4a24-a888-06fc5f4e5fe6","Type":"ContainerStarted","Data":"972a3a79c2a5586e86a3a151a4a0a9596b4f13f5e60f012597acef34460ade4e"}
Jul 07 11:12:04.313500 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:04.313473    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-9pdtd" event={"ID":"df6d16a6-2593-4477-ae7a-eca848817f34","Type":"ContainerStarted","Data":"660c9e1435cf6ddca5b103c6a1bc20eb8326666afb605c8c12230698c5c8f49b"}
Jul 07 11:12:04.314884 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:04.314840    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-px74n" podStartSLOduration=67.478555874 podStartE2EDuration="1m13.314823756s" totalImagesPullingTime="5.836267882s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:10:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:12:04.314465221 +0000 UTC m=+74.024361609" watchObservedRunningTime="2026-07-07 11:12:04.314823756 +0000 UTC m=+74.024720137"
Jul 07 11:12:04.338979 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:04.338755    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-5dc4cf6785-bflm8" podStartSLOduration=36.338731473 podStartE2EDuration="36.338731473s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:11:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:12:04.336727045 +0000 UTC m=+74.046623435" watchObservedRunningTime="2026-07-07 11:12:04.338731473 +0000 UTC m=+74.048627862"
Jul 07 11:12:04.362715 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:04.360806    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-8kjb7" podStartSLOduration=17.912039674 podStartE2EDuration="27.360784026s" totalImagesPullingTime="9.448744352s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:11:37 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:12:04.360707281 +0000 UTC m=+74.070603670" watchObservedRunningTime="2026-07-07 11:12:04.360784026 +0000 UTC m=+74.070680408"
Jul 07 11:12:04.389290 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:04.389228    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-64n4d" podStartSLOduration=33.425244856 podStartE2EDuration="38.389205837s" totalImagesPullingTime="4.963960981s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:11:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:12:04.388711586 +0000 UTC m=+74.098607973" watchObservedRunningTime="2026-07-07 11:12:04.389205837 +0000 UTC m=+74.099102226"
Jul 07 11:12:04.458022 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:04.457545    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79dc7c57c7-xgtwp"]
Jul 07 11:12:04.685139 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:04.685105    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-5dc4cf6785-bflm8"
Jul 07 11:12:04.688171 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:04.688126    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-5dc4cf6785-bflm8"
Jul 07 11:12:04.761265 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:12:04.761220    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2e8f3fdb_cc2c_4305_a9e3_c2b7ec951dd7.slice/crio-a6d890f2ef343e33b48dd858f472fe583efed969bbc8af8c6a46f69e98d4f756 WatchSource:0}: Error finding container a6d890f2ef343e33b48dd858f472fe583efed969bbc8af8c6a46f69e98d4f756: Status 404 returned error can't find the container with id a6d890f2ef343e33b48dd858f472fe583efed969bbc8af8c6a46f69e98d4f756
Jul 07 11:12:05.318410 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:05.318369    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79dc7c57c7-xgtwp" event={"ID":"2e8f3fdb-cc2c-4305-a9e3-c2b7ec951dd7","Type":"ContainerStarted","Data":"a6d890f2ef343e33b48dd858f472fe583efed969bbc8af8c6a46f69e98d4f756"}
Jul 07 11:12:05.320124 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:05.320094    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-vj4hq" event={"ID":"f9ec8255-2877-4c6a-b248-b50d64978f7a","Type":"ContainerStarted","Data":"1f379591427abdd04456060185842008221dfecbedfbd091dc2f8e6110e9ac3f"}
Jul 07 11:12:05.320246 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:05.320179    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-5dc4cf6785-bflm8"
Jul 07 11:12:05.321549 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:05.321528    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-5dc4cf6785-bflm8"
Jul 07 11:12:05.364583 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:05.364518    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-vj4hq" podStartSLOduration=36.22250257 podStartE2EDuration="37.36449565s" totalImagesPullingTime="1.14199308s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:11:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:12:05.340485602 +0000 UTC m=+75.050381990" watchObservedRunningTime="2026-07-07 11:12:05.36449565 +0000 UTC m=+75.074392039"
Jul 07 11:12:07.333116 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:07.333074    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-lmnhq" event={"ID":"63454a74-dfe7-497f-9f3d-0725bc111250","Type":"ContainerStarted","Data":"f3284796e78e352cda538032221d71ab031476ee409a78f67419c5c13689560d"}
Jul 07 11:12:07.333588 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:07.333147    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-lmnhq" event={"ID":"63454a74-dfe7-497f-9f3d-0725bc111250","Type":"ContainerStarted","Data":"8e70c11c3854542784886cbe7c163720ff36a61d07ccb9421be940b01d56b3b7"}
Jul 07 11:12:07.335708 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:07.335650    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-9pdtd" event={"ID":"df6d16a6-2593-4477-ae7a-eca848817f34","Type":"ContainerStarted","Data":"97a9cbba28084da7452cbb93ecf2ab0aae7b80656924a8505184eac38363d604"}
Jul 07 11:12:07.361274 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:07.361213    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-lmnhq" podStartSLOduration=36.402928658 podStartE2EDuration="39.361197157s" totalImagesPullingTime="2.958268499s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:11:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:12:07.359233771 +0000 UTC m=+77.069130173" watchObservedRunningTime="2026-07-07 11:12:07.361197157 +0000 UTC m=+77.071093546"
Jul 07 11:12:10.347946 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:10.347904    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79dc7c57c7-xgtwp" event={"ID":"2e8f3fdb-cc2c-4305-a9e3-c2b7ec951dd7","Type":"ContainerStarted","Data":"afac1e45067dbad01720b242915163be252e40b6685a6a9389f265cca73797a8"}
Jul 07 11:12:10.348377 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:10.347958    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79dc7c57c7-xgtwp" event={"ID":"2e8f3fdb-cc2c-4305-a9e3-c2b7ec951dd7","Type":"ContainerStarted","Data":"a439f86dcba75c78307a7c06d9a8ae82f40a6f74128a1d652d47b0b1655a0c65"}
Jul 07 11:12:10.348377 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:10.347971    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79dc7c57c7-xgtwp" event={"ID":"2e8f3fdb-cc2c-4305-a9e3-c2b7ec951dd7","Type":"ContainerStarted","Data":"ed354203d35d35d4101f780e84c50914831e192ea5e0c4ebe2c191dfa84978dc"}
Jul 07 11:12:10.369273 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:10.369220    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79dc7c57c7-xgtwp" podStartSLOduration=2.233061855 podStartE2EDuration="7.369203969s" totalImagesPullingTime="5.136142114s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:12:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:12:10.36755751 +0000 UTC m=+80.077453909" watchObservedRunningTime="2026-07-07 11:12:10.369203969 +0000 UTC m=+80.079100355"
Jul 07 11:12:10.370024 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:10.369994    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-9pdtd" podStartSLOduration=39.562927868 podStartE2EDuration="42.369986782s" totalImagesPullingTime="2.807058914s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:11:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:12:07.385455341 +0000 UTC m=+77.095351741" watchObservedRunningTime="2026-07-07 11:12:10.369986782 +0000 UTC m=+80.079883231"
Jul 07 11:12:10.816447 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:10.816412    2570 scope.go:117] "RemoveContainer" containerID="248c742829f41d119f62e9f783a582c6e1b35a6810ab916cc2a3cdc51ee2c9a4"
Jul 07 11:12:10.816645 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:12:10.816611    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-4v7v8_openshift-console-operator(c8c17c90-3e07-4530-8411-d59cef9a1aff)\"" pod="openshift-console-operator/console-operator-68b757fb67-4v7v8" podUID="c8c17c90-3e07-4530-8411-d59cef9a1aff"
Jul 07 11:12:14.322788 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:14.322673    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-64n4d"
Jul 07 11:12:18.230131 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:18.230094    2570 patch_prober.go:28] interesting pod/image-registry-9bb4d966c-zfwhd container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 07 11:12:18.230534 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:18.230158    2570 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-9bb4d966c-zfwhd" podUID="cc075655-13c0-4026-9ed2-8c4cc2a71dab" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 11:12:18.757681 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:18.757645    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-5gjjf"]
Jul 07 11:12:18.761392 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:18.761372    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5gjjf"
Jul 07 11:12:18.764855 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:18.764812    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 07 11:12:18.764855 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:18.764845    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-666wg\""
Jul 07 11:12:18.765038 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:18.764817    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 07 11:12:18.765038 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:18.764820    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 07 11:12:18.766028 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:18.766009    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 07 11:12:18.860420 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:18.860382    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/773efbaa-d3a5-4339-96e0-432ce18032ca-metrics-client-ca\") pod \"node-exporter-5gjjf\" (UID: \"773efbaa-d3a5-4339-96e0-432ce18032ca\") " pod="openshift-monitoring/node-exporter-5gjjf"
Jul 07 11:12:18.860420 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:18.860416    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5qfp\" (UniqueName: \"kubernetes.io/projected/773efbaa-d3a5-4339-96e0-432ce18032ca-kube-api-access-c5qfp\") pod \"node-exporter-5gjjf\" (UID: \"773efbaa-d3a5-4339-96e0-432ce18032ca\") " pod="openshift-monitoring/node-exporter-5gjjf"
Jul 07 11:12:18.860624 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:18.860486    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/773efbaa-d3a5-4339-96e0-432ce18032ca-sys\") pod \"node-exporter-5gjjf\" (UID: \"773efbaa-d3a5-4339-96e0-432ce18032ca\") " pod="openshift-monitoring/node-exporter-5gjjf"
Jul 07 11:12:18.860624 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:18.860517    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/773efbaa-d3a5-4339-96e0-432ce18032ca-node-exporter-tls\") pod \"node-exporter-5gjjf\" (UID: \"773efbaa-d3a5-4339-96e0-432ce18032ca\") " pod="openshift-monitoring/node-exporter-5gjjf"
Jul 07 11:12:18.860624 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:18.860539    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/773efbaa-d3a5-4339-96e0-432ce18032ca-node-exporter-wtmp\") pod \"node-exporter-5gjjf\" (UID: \"773efbaa-d3a5-4339-96e0-432ce18032ca\") " pod="openshift-monitoring/node-exporter-5gjjf"
Jul 07 11:12:18.860624 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:18.860568    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/773efbaa-d3a5-4339-96e0-432ce18032ca-node-exporter-accelerators-collector-config\") pod \"node-exporter-5gjjf\" (UID: \"773efbaa-d3a5-4339-96e0-432ce18032ca\") " pod="openshift-monitoring/node-exporter-5gjjf"
Jul 07 11:12:18.860624 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:18.860596    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/773efbaa-d3a5-4339-96e0-432ce18032ca-root\") pod \"node-exporter-5gjjf\" (UID: \"773efbaa-d3a5-4339-96e0-432ce18032ca\") " pod="openshift-monitoring/node-exporter-5gjjf"
Jul 07 11:12:18.860624 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:18.860616    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/773efbaa-d3a5-4339-96e0-432ce18032ca-node-exporter-textfile\") pod \"node-exporter-5gjjf\" (UID: \"773efbaa-d3a5-4339-96e0-432ce18032ca\") " pod="openshift-monitoring/node-exporter-5gjjf"
Jul 07 11:12:18.860822 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:18.860635    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/773efbaa-d3a5-4339-96e0-432ce18032ca-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5gjjf\" (UID: \"773efbaa-d3a5-4339-96e0-432ce18032ca\") " pod="openshift-monitoring/node-exporter-5gjjf"
Jul 07 11:12:18.960986 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:18.960947    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/773efbaa-d3a5-4339-96e0-432ce18032ca-sys\") pod \"node-exporter-5gjjf\" (UID: \"773efbaa-d3a5-4339-96e0-432ce18032ca\") " pod="openshift-monitoring/node-exporter-5gjjf"
Jul 07 11:12:18.960986 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:18.960988    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/773efbaa-d3a5-4339-96e0-432ce18032ca-node-exporter-tls\") pod \"node-exporter-5gjjf\" (UID: \"773efbaa-d3a5-4339-96e0-432ce18032ca\") " pod="openshift-monitoring/node-exporter-5gjjf"
Jul 07 11:12:18.961249 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:18.961012    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/773efbaa-d3a5-4339-96e0-432ce18032ca-node-exporter-wtmp\") pod \"node-exporter-5gjjf\" (UID: \"773efbaa-d3a5-4339-96e0-432ce18032ca\") " pod="openshift-monitoring/node-exporter-5gjjf"
Jul 07 11:12:18.961249 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:18.961046    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/773efbaa-d3a5-4339-96e0-432ce18032ca-node-exporter-accelerators-collector-config\") pod \"node-exporter-5gjjf\" (UID: \"773efbaa-d3a5-4339-96e0-432ce18032ca\") " pod="openshift-monitoring/node-exporter-5gjjf"
Jul 07 11:12:18.961249 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:18.961065    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/773efbaa-d3a5-4339-96e0-432ce18032ca-sys\") pod \"node-exporter-5gjjf\" (UID: \"773efbaa-d3a5-4339-96e0-432ce18032ca\") " pod="openshift-monitoring/node-exporter-5gjjf"
Jul 07 11:12:18.961249 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:18.961088    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/773efbaa-d3a5-4339-96e0-432ce18032ca-root\") pod \"node-exporter-5gjjf\" (UID: \"773efbaa-d3a5-4339-96e0-432ce18032ca\") " pod="openshift-monitoring/node-exporter-5gjjf"
Jul 07 11:12:18.961249 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:18.961120    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/773efbaa-d3a5-4339-96e0-432ce18032ca-node-exporter-textfile\") pod \"node-exporter-5gjjf\" (UID: \"773efbaa-d3a5-4339-96e0-432ce18032ca\") " pod="openshift-monitoring/node-exporter-5gjjf"
Jul 07 11:12:18.961249 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:18.961146    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/773efbaa-d3a5-4339-96e0-432ce18032ca-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5gjjf\" (UID: \"773efbaa-d3a5-4339-96e0-432ce18032ca\") " pod="openshift-monitoring/node-exporter-5gjjf"
Jul 07 11:12:18.961249 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:18.961178    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/773efbaa-d3a5-4339-96e0-432ce18032ca-metrics-client-ca\") pod \"node-exporter-5gjjf\" (UID: \"773efbaa-d3a5-4339-96e0-432ce18032ca\") " pod="openshift-monitoring/node-exporter-5gjjf"
Jul 07 11:12:18.961249 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:18.961202    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5qfp\" (UniqueName: \"kubernetes.io/projected/773efbaa-d3a5-4339-96e0-432ce18032ca-kube-api-access-c5qfp\") pod \"node-exporter-5gjjf\" (UID: \"773efbaa-d3a5-4339-96e0-432ce18032ca\") " pod="openshift-monitoring/node-exporter-5gjjf"
Jul 07 11:12:18.961249 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:18.961217    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/773efbaa-d3a5-4339-96e0-432ce18032ca-node-exporter-wtmp\") pod \"node-exporter-5gjjf\" (UID: \"773efbaa-d3a5-4339-96e0-432ce18032ca\") " pod="openshift-monitoring/node-exporter-5gjjf"
Jul 07 11:12:18.961694 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:18.961282    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/773efbaa-d3a5-4339-96e0-432ce18032ca-root\") pod \"node-exporter-5gjjf\" (UID: \"773efbaa-d3a5-4339-96e0-432ce18032ca\") " pod="openshift-monitoring/node-exporter-5gjjf"
Jul 07 11:12:18.961694 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:18.961540    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/773efbaa-d3a5-4339-96e0-432ce18032ca-node-exporter-textfile\") pod \"node-exporter-5gjjf\" (UID: \"773efbaa-d3a5-4339-96e0-432ce18032ca\") " pod="openshift-monitoring/node-exporter-5gjjf"
Jul 07 11:12:18.961888 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:18.961831    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/773efbaa-d3a5-4339-96e0-432ce18032ca-node-exporter-accelerators-collector-config\") pod \"node-exporter-5gjjf\" (UID: \"773efbaa-d3a5-4339-96e0-432ce18032ca\") " pod="openshift-monitoring/node-exporter-5gjjf"
Jul 07 11:12:18.961888 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:18.961852    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/773efbaa-d3a5-4339-96e0-432ce18032ca-metrics-client-ca\") pod \"node-exporter-5gjjf\" (UID: \"773efbaa-d3a5-4339-96e0-432ce18032ca\") " pod="openshift-monitoring/node-exporter-5gjjf"
Jul 07 11:12:18.963587 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:18.963566    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/773efbaa-d3a5-4339-96e0-432ce18032ca-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5gjjf\" (UID: \"773efbaa-d3a5-4339-96e0-432ce18032ca\") " pod="openshift-monitoring/node-exporter-5gjjf"
Jul 07 11:12:18.963690 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:18.963655    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/773efbaa-d3a5-4339-96e0-432ce18032ca-node-exporter-tls\") pod \"node-exporter-5gjjf\" (UID: \"773efbaa-d3a5-4339-96e0-432ce18032ca\") " pod="openshift-monitoring/node-exporter-5gjjf"
Jul 07 11:12:18.970802 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:18.970779    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5qfp\" (UniqueName: \"kubernetes.io/projected/773efbaa-d3a5-4339-96e0-432ce18032ca-kube-api-access-c5qfp\") pod \"node-exporter-5gjjf\" (UID: \"773efbaa-d3a5-4339-96e0-432ce18032ca\") " pod="openshift-monitoring/node-exporter-5gjjf"
Jul 07 11:12:19.070521 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:19.070442    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5gjjf"
Jul 07 11:12:19.082220 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:12:19.082182    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod773efbaa_d3a5_4339_96e0_432ce18032ca.slice/crio-edd7bd74d9ecfc8fe5bbc352392666a6ee85215a400a40c782ebe6e34ce827c3 WatchSource:0}: Error finding container edd7bd74d9ecfc8fe5bbc352392666a6ee85215a400a40c782ebe6e34ce827c3: Status 404 returned error can't find the container with id edd7bd74d9ecfc8fe5bbc352392666a6ee85215a400a40c782ebe6e34ce827c3
Jul 07 11:12:19.381637 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:19.381559    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5gjjf" event={"ID":"773efbaa-d3a5-4339-96e0-432ce18032ca","Type":"ContainerStarted","Data":"edd7bd74d9ecfc8fe5bbc352392666a6ee85215a400a40c782ebe6e34ce827c3"}
Jul 07 11:12:20.282079 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:20.282046    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-9bb4d966c-zfwhd"
Jul 07 11:12:20.389913 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:20.389873    2570 generic.go:356] "Generic (PLEG): container finished" podID="773efbaa-d3a5-4339-96e0-432ce18032ca" containerID="2069ecda66809081d3b5bd7e93f529df91e61711008752249c176c7a3d3da229" exitCode=0
Jul 07 11:12:20.390388 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:20.389962    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5gjjf" event={"ID":"773efbaa-d3a5-4339-96e0-432ce18032ca","Type":"ContainerDied","Data":"2069ecda66809081d3b5bd7e93f529df91e61711008752249c176c7a3d3da229"}
Jul 07 11:12:21.395683 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:21.395646    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5gjjf" event={"ID":"773efbaa-d3a5-4339-96e0-432ce18032ca","Type":"ContainerStarted","Data":"ab5ed7631f697cec3aa95ee6e7b00673f63d380ecf2ab812d9a7d17252ad0200"}
Jul 07 11:12:21.396075 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:21.395703    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5gjjf" event={"ID":"773efbaa-d3a5-4339-96e0-432ce18032ca","Type":"ContainerStarted","Data":"34a98711cb6e10c9a1a30ae1a6cedd11cba61195c644f8d05404306ca3ba184b"}
Jul 07 11:12:21.423617 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:21.423568    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-5gjjf" podStartSLOduration=2.454398051 podStartE2EDuration="3.423552806s" totalImagesPullingTime="894.539814ms" totalInitContainerRuntime="74.614941ms" isStatefulPod=true podCreationTimestamp="2026-07-07 11:12:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:12:21.421316395 +0000 UTC m=+91.131212782" watchObservedRunningTime="2026-07-07 11:12:21.423552806 +0000 UTC m=+91.133449192"
Jul 07 11:12:23.815690 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:23.815656    2570 scope.go:117] "RemoveContainer" containerID="248c742829f41d119f62e9f783a582c6e1b35a6810ab916cc2a3cdc51ee2c9a4"
Jul 07 11:12:24.413535 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:24.413504    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4v7v8_c8c17c90-3e07-4530-8411-d59cef9a1aff/console-operator/2.log"
Jul 07 11:12:24.413719 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:24.413554    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-4v7v8" event={"ID":"c8c17c90-3e07-4530-8411-d59cef9a1aff","Type":"ContainerStarted","Data":"e4a5ed08de962e933afa5482b31faf95174cab1700e6bd7275965ea84c1a38b1"}
Jul 07 11:12:24.413854 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:24.413832    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-4v7v8"
Jul 07 11:12:24.424179 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:24.424138    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-4v7v8"
Jul 07 11:12:24.442607 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:24.442549    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-4v7v8" podStartSLOduration=48.780143595 podStartE2EDuration="56.442534435s" totalImagesPullingTime="7.66239084s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:11:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:12:24.441215434 +0000 UTC m=+94.151111821" watchObservedRunningTime="2026-07-07 11:12:24.442534435 +0000 UTC m=+94.152430822"
Jul 07 11:12:28.480881 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:28.480849    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-9bb4d966c-zfwhd"]
Jul 07 11:12:29.271026 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:29.270989    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-n5mhp"
Jul 07 11:12:53.501930 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:53.501868    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-9bb4d966c-zfwhd" podUID="cc075655-13c0-4026-9ed2-8c4cc2a71dab" containerName="registry" containerID="cri-o://2811c01af62ae5f83b021a5c84cdcd050f3d6d19bc3ab55395cb8acf59c0c269" gracePeriod=30
Jul 07 11:12:53.523418 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:53.523390    2570 generic.go:356] "Generic (PLEG): container finished" podID="192f418c-f6e6-4775-876f-ef3d4f11b265" containerID="c04d291934a232bc520c0285edf2a4deccfb52854997e1b64d1c219db9f2b6e9" exitCode=0
Jul 07 11:12:53.523531 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:53.523459    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-95lgm" event={"ID":"192f418c-f6e6-4775-876f-ef3d4f11b265","Type":"ContainerDied","Data":"c04d291934a232bc520c0285edf2a4deccfb52854997e1b64d1c219db9f2b6e9"}
Jul 07 11:12:53.523788 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:53.523775    2570 scope.go:117] "RemoveContainer" containerID="c04d291934a232bc520c0285edf2a4deccfb52854997e1b64d1c219db9f2b6e9"
Jul 07 11:12:53.766668 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:53.766604    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9bb4d966c-zfwhd"
Jul 07 11:12:53.858374 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:53.858345    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cc075655-13c0-4026-9ed2-8c4cc2a71dab-installation-pull-secrets\") pod \"cc075655-13c0-4026-9ed2-8c4cc2a71dab\" (UID: \"cc075655-13c0-4026-9ed2-8c4cc2a71dab\") "
Jul 07 11:12:53.858569 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:53.858397    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cc075655-13c0-4026-9ed2-8c4cc2a71dab-bound-sa-token\") pod \"cc075655-13c0-4026-9ed2-8c4cc2a71dab\" (UID: \"cc075655-13c0-4026-9ed2-8c4cc2a71dab\") "
Jul 07 11:12:53.858569 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:53.858420    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc075655-13c0-4026-9ed2-8c4cc2a71dab-registry-tls\") pod \"cc075655-13c0-4026-9ed2-8c4cc2a71dab\" (UID: \"cc075655-13c0-4026-9ed2-8c4cc2a71dab\") "
Jul 07 11:12:53.858569 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:53.858441    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l8bz7\" (UniqueName: \"kubernetes.io/projected/cc075655-13c0-4026-9ed2-8c4cc2a71dab-kube-api-access-l8bz7\") pod \"cc075655-13c0-4026-9ed2-8c4cc2a71dab\" (UID: \"cc075655-13c0-4026-9ed2-8c4cc2a71dab\") "
Jul 07 11:12:53.858742 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:53.858577    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cc075655-13c0-4026-9ed2-8c4cc2a71dab-image-registry-private-configuration\") pod \"cc075655-13c0-4026-9ed2-8c4cc2a71dab\" (UID: \"cc075655-13c0-4026-9ed2-8c4cc2a71dab\") "
Jul 07 11:12:53.858742 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:53.858631    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cc075655-13c0-4026-9ed2-8c4cc2a71dab-trusted-ca\") pod \"cc075655-13c0-4026-9ed2-8c4cc2a71dab\" (UID: \"cc075655-13c0-4026-9ed2-8c4cc2a71dab\") "
Jul 07 11:12:53.858742 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:53.858680    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cc075655-13c0-4026-9ed2-8c4cc2a71dab-ca-trust-extracted\") pod \"cc075655-13c0-4026-9ed2-8c4cc2a71dab\" (UID: \"cc075655-13c0-4026-9ed2-8c4cc2a71dab\") "
Jul 07 11:12:53.858742 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:53.858728    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cc075655-13c0-4026-9ed2-8c4cc2a71dab-registry-certificates\") pod \"cc075655-13c0-4026-9ed2-8c4cc2a71dab\" (UID: \"cc075655-13c0-4026-9ed2-8c4cc2a71dab\") "
Jul 07 11:12:53.859276 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:53.859209    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc075655-13c0-4026-9ed2-8c4cc2a71dab-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "cc075655-13c0-4026-9ed2-8c4cc2a71dab" (UID: "cc075655-13c0-4026-9ed2-8c4cc2a71dab"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 11:12:53.859276 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:53.859254    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc075655-13c0-4026-9ed2-8c4cc2a71dab-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "cc075655-13c0-4026-9ed2-8c4cc2a71dab" (UID: "cc075655-13c0-4026-9ed2-8c4cc2a71dab"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 11:12:53.861110 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:53.861063    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc075655-13c0-4026-9ed2-8c4cc2a71dab-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "cc075655-13c0-4026-9ed2-8c4cc2a71dab" (UID: "cc075655-13c0-4026-9ed2-8c4cc2a71dab"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:12:53.861425 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:53.861364    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc075655-13c0-4026-9ed2-8c4cc2a71dab-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "cc075655-13c0-4026-9ed2-8c4cc2a71dab" (UID: "cc075655-13c0-4026-9ed2-8c4cc2a71dab"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 11:12:53.861528 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:53.861430    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc075655-13c0-4026-9ed2-8c4cc2a71dab-kube-api-access-l8bz7" (OuterVolumeSpecName: "kube-api-access-l8bz7") pod "cc075655-13c0-4026-9ed2-8c4cc2a71dab" (UID: "cc075655-13c0-4026-9ed2-8c4cc2a71dab"). InnerVolumeSpecName "kube-api-access-l8bz7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:12:53.861609 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:53.861578    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc075655-13c0-4026-9ed2-8c4cc2a71dab-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "cc075655-13c0-4026-9ed2-8c4cc2a71dab" (UID: "cc075655-13c0-4026-9ed2-8c4cc2a71dab"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 11:12:53.861852 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:53.861828    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc075655-13c0-4026-9ed2-8c4cc2a71dab-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "cc075655-13c0-4026-9ed2-8c4cc2a71dab" (UID: "cc075655-13c0-4026-9ed2-8c4cc2a71dab"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:12:53.870783 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:53.870758    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc075655-13c0-4026-9ed2-8c4cc2a71dab-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "cc075655-13c0-4026-9ed2-8c4cc2a71dab" (UID: "cc075655-13c0-4026-9ed2-8c4cc2a71dab"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:12:53.959579 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:53.959545    2570 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cc075655-13c0-4026-9ed2-8c4cc2a71dab-ca-trust-extracted\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:12:53.959579 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:53.959572    2570 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cc075655-13c0-4026-9ed2-8c4cc2a71dab-registry-certificates\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:12:53.959579 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:53.959583    2570 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cc075655-13c0-4026-9ed2-8c4cc2a71dab-installation-pull-secrets\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:12:53.959579 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:53.959591    2570 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cc075655-13c0-4026-9ed2-8c4cc2a71dab-bound-sa-token\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:12:53.959849 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:53.959600    2570 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc075655-13c0-4026-9ed2-8c4cc2a71dab-registry-tls\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:12:53.959849 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:53.959609    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l8bz7\" (UniqueName: \"kubernetes.io/projected/cc075655-13c0-4026-9ed2-8c4cc2a71dab-kube-api-access-l8bz7\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:12:53.959849 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:53.959619    2570 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cc075655-13c0-4026-9ed2-8c4cc2a71dab-image-registry-private-configuration\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:12:53.959849 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:53.959627    2570 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cc075655-13c0-4026-9ed2-8c4cc2a71dab-trusted-ca\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:12:54.529579 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:54.529513    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-95lgm" event={"ID":"192f418c-f6e6-4775-876f-ef3d4f11b265","Type":"ContainerStarted","Data":"c6a11a7d0fdc34f76d4860641145ab94c9e7b59374e1dcc7b3f21be766eda693"}
Jul 07 11:12:54.530951 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:54.530928    2570 generic.go:356] "Generic (PLEG): container finished" podID="cc075655-13c0-4026-9ed2-8c4cc2a71dab" containerID="2811c01af62ae5f83b021a5c84cdcd050f3d6d19bc3ab55395cb8acf59c0c269" exitCode=0
Jul 07 11:12:54.531049 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:54.530976    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9bb4d966c-zfwhd"
Jul 07 11:12:54.531049 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:54.531012    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9bb4d966c-zfwhd" event={"ID":"cc075655-13c0-4026-9ed2-8c4cc2a71dab","Type":"ContainerDied","Data":"2811c01af62ae5f83b021a5c84cdcd050f3d6d19bc3ab55395cb8acf59c0c269"}
Jul 07 11:12:54.531117 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:54.531056    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9bb4d966c-zfwhd" event={"ID":"cc075655-13c0-4026-9ed2-8c4cc2a71dab","Type":"ContainerDied","Data":"1af2ace849a357649917c2fe26f54cf1f71e4e755ce69a9baa404bc076a52677"}
Jul 07 11:12:54.531117 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:54.531076    2570 scope.go:117] "RemoveContainer" containerID="2811c01af62ae5f83b021a5c84cdcd050f3d6d19bc3ab55395cb8acf59c0c269"
Jul 07 11:12:54.542965 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:54.542947    2570 scope.go:117] "RemoveContainer" containerID="2811c01af62ae5f83b021a5c84cdcd050f3d6d19bc3ab55395cb8acf59c0c269"
Jul 07 11:12:54.543222 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:12:54.543202    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2811c01af62ae5f83b021a5c84cdcd050f3d6d19bc3ab55395cb8acf59c0c269\": container with ID starting with 2811c01af62ae5f83b021a5c84cdcd050f3d6d19bc3ab55395cb8acf59c0c269 not found: ID does not exist" containerID="2811c01af62ae5f83b021a5c84cdcd050f3d6d19bc3ab55395cb8acf59c0c269"
Jul 07 11:12:54.543284 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:54.543231    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2811c01af62ae5f83b021a5c84cdcd050f3d6d19bc3ab55395cb8acf59c0c269"} err="failed to get container status \"2811c01af62ae5f83b021a5c84cdcd050f3d6d19bc3ab55395cb8acf59c0c269\": rpc error: code = NotFound desc = could not find container \"2811c01af62ae5f83b021a5c84cdcd050f3d6d19bc3ab55395cb8acf59c0c269\": container with ID starting with 2811c01af62ae5f83b021a5c84cdcd050f3d6d19bc3ab55395cb8acf59c0c269 not found: ID does not exist"
Jul 07 11:12:54.575316 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:54.575285    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-9bb4d966c-zfwhd"]
Jul 07 11:12:54.581242 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:54.581215    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-9bb4d966c-zfwhd"]
Jul 07 11:12:54.819460 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:12:54.819379    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc075655-13c0-4026-9ed2-8c4cc2a71dab" path="/var/lib/kubelet/pods/cc075655-13c0-4026-9ed2-8c4cc2a71dab/volumes"
Jul 07 11:13:02.560904 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:13:02.560870    2570 generic.go:356] "Generic (PLEG): container finished" podID="6387d3da-1290-437c-8fdc-2422d49dac72" containerID="001aeae498a26940179e64df14dc5a0862488a4c82b9897c7570f19542670d31" exitCode=0
Jul 07 11:13:02.561450 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:13:02.560950    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-wffz6" event={"ID":"6387d3da-1290-437c-8fdc-2422d49dac72","Type":"ContainerDied","Data":"001aeae498a26940179e64df14dc5a0862488a4c82b9897c7570f19542670d31"}
Jul 07 11:13:02.561450 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:13:02.561444    2570 scope.go:117] "RemoveContainer" containerID="001aeae498a26940179e64df14dc5a0862488a4c82b9897c7570f19542670d31"
Jul 07 11:13:03.565825 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:13:03.565786    2570 generic.go:356] "Generic (PLEG): container finished" podID="7fba6280-ffbe-4a71-98f3-3432b0584697" containerID="052701c116eb9b9b1cec2e45cd595c994407f937caa13282328f227cf88329d9" exitCode=0
Jul 07 11:13:03.566273 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:13:03.565860    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6h7pw" event={"ID":"7fba6280-ffbe-4a71-98f3-3432b0584697","Type":"ContainerDied","Data":"052701c116eb9b9b1cec2e45cd595c994407f937caa13282328f227cf88329d9"}
Jul 07 11:13:03.566273 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:13:03.566208    2570 scope.go:117] "RemoveContainer" containerID="052701c116eb9b9b1cec2e45cd595c994407f937caa13282328f227cf88329d9"
Jul 07 11:13:03.567891 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:13:03.567872    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-wffz6" event={"ID":"6387d3da-1290-437c-8fdc-2422d49dac72","Type":"ContainerStarted","Data":"7e3b6a8c19469da52cd210694cf83a170fcf6110e0540ca91169e9aff2743965"}
Jul 07 11:13:04.573119 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:13:04.573087    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6h7pw" event={"ID":"7fba6280-ffbe-4a71-98f3-3432b0584697","Type":"ContainerStarted","Data":"c2b694ed7251c622c324542723cacf3ecae8dab8c86f05ffd3e81805a87b3d37"}
Jul 07 11:15:40.072973 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:15:40.072935    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-bxzht"]
Jul 07 11:15:40.073403 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:15:40.073339    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc075655-13c0-4026-9ed2-8c4cc2a71dab" containerName="registry"
Jul 07 11:15:40.073403 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:15:40.073356    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc075655-13c0-4026-9ed2-8c4cc2a71dab" containerName="registry"
Jul 07 11:15:40.073488 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:15:40.073433    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc075655-13c0-4026-9ed2-8c4cc2a71dab" containerName="registry"
Jul 07 11:15:40.076453 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:15:40.076434    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-bxzht"
Jul 07 11:15:40.081905 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:15:40.081887    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 07 11:15:40.082048 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:15:40.082022    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 07 11:15:40.082112 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:15:40.082093    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 07 11:15:40.082849 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:15:40.082724    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-6xcdv\""
Jul 07 11:15:40.082975 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:15:40.082958    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 07 11:15:40.104538 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:15:40.104510    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-bxzht"]
Jul 07 11:15:40.236880 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:15:40.236843    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/948ae785-e791-40b1-a7c8-029a6e9e3d8e-manifests\") pod \"opendatahub-operator-controller-manager-6dc7dd79cd-bxzht\" (UID: \"948ae785-e791-40b1-a7c8-029a6e9e3d8e\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-bxzht"
Jul 07 11:15:40.237069 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:15:40.236900    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tx6rs\" (UniqueName: \"kubernetes.io/projected/948ae785-e791-40b1-a7c8-029a6e9e3d8e-kube-api-access-tx6rs\") pod \"opendatahub-operator-controller-manager-6dc7dd79cd-bxzht\" (UID: \"948ae785-e791-40b1-a7c8-029a6e9e3d8e\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-bxzht"
Jul 07 11:15:40.237069 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:15:40.236922    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/948ae785-e791-40b1-a7c8-029a6e9e3d8e-tmp\") pod \"opendatahub-operator-controller-manager-6dc7dd79cd-bxzht\" (UID: \"948ae785-e791-40b1-a7c8-029a6e9e3d8e\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-bxzht"
Jul 07 11:15:40.237069 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:15:40.236943    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/948ae785-e791-40b1-a7c8-029a6e9e3d8e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6dc7dd79cd-bxzht\" (UID: \"948ae785-e791-40b1-a7c8-029a6e9e3d8e\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-bxzht"
Jul 07 11:15:40.237069 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:15:40.236961    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/948ae785-e791-40b1-a7c8-029a6e9e3d8e-webhook-cert\") pod \"opendatahub-operator-controller-manager-6dc7dd79cd-bxzht\" (UID: \"948ae785-e791-40b1-a7c8-029a6e9e3d8e\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-bxzht"
Jul 07 11:15:40.337922 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:15:40.337826    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/948ae785-e791-40b1-a7c8-029a6e9e3d8e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6dc7dd79cd-bxzht\" (UID: \"948ae785-e791-40b1-a7c8-029a6e9e3d8e\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-bxzht"
Jul 07 11:15:40.337922 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:15:40.337900    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/948ae785-e791-40b1-a7c8-029a6e9e3d8e-webhook-cert\") pod \"opendatahub-operator-controller-manager-6dc7dd79cd-bxzht\" (UID: \"948ae785-e791-40b1-a7c8-029a6e9e3d8e\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-bxzht"
Jul 07 11:15:40.338122 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:15:40.337953    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/948ae785-e791-40b1-a7c8-029a6e9e3d8e-manifests\") pod \"opendatahub-operator-controller-manager-6dc7dd79cd-bxzht\" (UID: \"948ae785-e791-40b1-a7c8-029a6e9e3d8e\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-bxzht"
Jul 07 11:15:40.338122 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:15:40.338011    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tx6rs\" (UniqueName: \"kubernetes.io/projected/948ae785-e791-40b1-a7c8-029a6e9e3d8e-kube-api-access-tx6rs\") pod \"opendatahub-operator-controller-manager-6dc7dd79cd-bxzht\" (UID: \"948ae785-e791-40b1-a7c8-029a6e9e3d8e\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-bxzht"
Jul 07 11:15:40.338122 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:15:40.338032    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/948ae785-e791-40b1-a7c8-029a6e9e3d8e-tmp\") pod \"opendatahub-operator-controller-manager-6dc7dd79cd-bxzht\" (UID: \"948ae785-e791-40b1-a7c8-029a6e9e3d8e\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-bxzht"
Jul 07 11:15:40.338461 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:15:40.338390    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/948ae785-e791-40b1-a7c8-029a6e9e3d8e-manifests\") pod \"opendatahub-operator-controller-manager-6dc7dd79cd-bxzht\" (UID: \"948ae785-e791-40b1-a7c8-029a6e9e3d8e\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-bxzht"
Jul 07 11:15:40.340295 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:15:40.340273    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/948ae785-e791-40b1-a7c8-029a6e9e3d8e-tmp\") pod \"opendatahub-operator-controller-manager-6dc7dd79cd-bxzht\" (UID: \"948ae785-e791-40b1-a7c8-029a6e9e3d8e\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-bxzht"
Jul 07 11:15:40.340457 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:15:40.340440    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/948ae785-e791-40b1-a7c8-029a6e9e3d8e-webhook-cert\") pod \"opendatahub-operator-controller-manager-6dc7dd79cd-bxzht\" (UID: \"948ae785-e791-40b1-a7c8-029a6e9e3d8e\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-bxzht"
Jul 07 11:15:40.340506 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:15:40.340450    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/948ae785-e791-40b1-a7c8-029a6e9e3d8e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6dc7dd79cd-bxzht\" (UID: \"948ae785-e791-40b1-a7c8-029a6e9e3d8e\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-bxzht"
Jul 07 11:15:40.373241 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:15:40.373211    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tx6rs\" (UniqueName: \"kubernetes.io/projected/948ae785-e791-40b1-a7c8-029a6e9e3d8e-kube-api-access-tx6rs\") pod \"opendatahub-operator-controller-manager-6dc7dd79cd-bxzht\" (UID: \"948ae785-e791-40b1-a7c8-029a6e9e3d8e\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-bxzht"
Jul 07 11:15:40.386077 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:15:40.386054    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-bxzht"
Jul 07 11:15:40.563090 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:15:40.563059    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-bxzht"]
Jul 07 11:15:40.565209 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:15:40.565173    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod948ae785_e791_40b1_a7c8_029a6e9e3d8e.slice/crio-e6419803c032ffe528ccba29b098812e066a86e4b069e8900ea7b562b87b12aa WatchSource:0}: Error finding container e6419803c032ffe528ccba29b098812e066a86e4b069e8900ea7b562b87b12aa: Status 404 returned error can't find the container with id e6419803c032ffe528ccba29b098812e066a86e4b069e8900ea7b562b87b12aa
Jul 07 11:15:41.145877 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:15:41.145838    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-bxzht" event={"ID":"948ae785-e791-40b1-a7c8-029a6e9e3d8e","Type":"ContainerStarted","Data":"e6419803c032ffe528ccba29b098812e066a86e4b069e8900ea7b562b87b12aa"}
Jul 07 11:15:44.165239 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:15:44.165146    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-bxzht" event={"ID":"948ae785-e791-40b1-a7c8-029a6e9e3d8e","Type":"ContainerStarted","Data":"6168dcad776d94eb4e63a16401e00c552a4058efc0590e8e33c720a21f615c9e"}
Jul 07 11:15:45.170997 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:15:45.170956    2570 generic.go:356] "Generic (PLEG): container finished" podID="948ae785-e791-40b1-a7c8-029a6e9e3d8e" containerID="6168dcad776d94eb4e63a16401e00c552a4058efc0590e8e33c720a21f615c9e" exitCode=0
Jul 07 11:15:45.171446 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:15:45.171047    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-bxzht" event={"ID":"948ae785-e791-40b1-a7c8-029a6e9e3d8e","Type":"ContainerDied","Data":"6168dcad776d94eb4e63a16401e00c552a4058efc0590e8e33c720a21f615c9e"}
Jul 07 11:15:46.176348 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:15:46.176226    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-bxzht" event={"ID":"948ae785-e791-40b1-a7c8-029a6e9e3d8e","Type":"ContainerStarted","Data":"3e714101b20ad061f0478053f0f6c1af74167d49a1f10f4c56cc231b3644f1dd"}
Jul 07 11:15:46.176786 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:15:46.176354    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-bxzht"
Jul 07 11:15:46.212716 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:15:46.212661    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-bxzht" podStartSLOduration=1.222373363 podStartE2EDuration="6.212645378s" totalImagesPullingTime="3.958394219s" totalInitContainerRuntime="1.031877796s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:15:40 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:15:46.210717554 +0000 UTC m=+295.920613942" watchObservedRunningTime="2026-07-07 11:15:46.212645378 +0000 UTC m=+295.922541765"
Jul 07 11:15:50.709771 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:15:50.709740    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4v7v8_c8c17c90-3e07-4530-8411-d59cef9a1aff/console-operator/2.log"
Jul 07 11:15:50.717258 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:15:50.717225    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4v7v8_c8c17c90-3e07-4530-8411-d59cef9a1aff/console-operator/2.log"
Jul 07 11:15:50.720697 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:15:50.720675    2570 kubelet.go:1648] "Image garbage collection succeeded"
Jul 07 11:15:57.182586 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:15:57.182548    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-bxzht"
Jul 07 11:15:59.952134 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:15:59.952099    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-qrhf8"]
Jul 07 11:15:59.955089 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:15:59.955073    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-qrhf8"
Jul 07 11:15:59.960092 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:15:59.960066    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-9m25b\""
Jul 07 11:15:59.960219 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:15:59.960140    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 07 11:15:59.969654 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:15:59.969626    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-qrhf8"]
Jul 07 11:15:59.986608 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:15:59.986578    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/85974022-4d26-4e54-bda4-3b39cce716cd-cert\") pod \"odh-model-controller-7998c67dc4-qrhf8\" (UID: \"85974022-4d26-4e54-bda4-3b39cce716cd\") " pod="opendatahub/odh-model-controller-7998c67dc4-qrhf8"
Jul 07 11:15:59.986744 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:15:59.986628    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27zpv\" (UniqueName: \"kubernetes.io/projected/85974022-4d26-4e54-bda4-3b39cce716cd-kube-api-access-27zpv\") pod \"odh-model-controller-7998c67dc4-qrhf8\" (UID: \"85974022-4d26-4e54-bda4-3b39cce716cd\") " pod="opendatahub/odh-model-controller-7998c67dc4-qrhf8"
Jul 07 11:16:00.087024 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:00.086985    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27zpv\" (UniqueName: \"kubernetes.io/projected/85974022-4d26-4e54-bda4-3b39cce716cd-kube-api-access-27zpv\") pod \"odh-model-controller-7998c67dc4-qrhf8\" (UID: \"85974022-4d26-4e54-bda4-3b39cce716cd\") " pod="opendatahub/odh-model-controller-7998c67dc4-qrhf8"
Jul 07 11:16:00.087186 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:00.087083    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/85974022-4d26-4e54-bda4-3b39cce716cd-cert\") pod \"odh-model-controller-7998c67dc4-qrhf8\" (UID: \"85974022-4d26-4e54-bda4-3b39cce716cd\") " pod="opendatahub/odh-model-controller-7998c67dc4-qrhf8"
Jul 07 11:16:00.087233 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:16:00.087188    2570 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 07 11:16:00.087266 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:16:00.087253    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85974022-4d26-4e54-bda4-3b39cce716cd-cert podName:85974022-4d26-4e54-bda4-3b39cce716cd nodeName:}" failed. No retries permitted until 2026-07-07 11:16:00.587234944 +0000 UTC m=+310.297131309 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/85974022-4d26-4e54-bda4-3b39cce716cd-cert") pod "odh-model-controller-7998c67dc4-qrhf8" (UID: "85974022-4d26-4e54-bda4-3b39cce716cd") : secret "odh-model-controller-webhook-cert" not found
Jul 07 11:16:00.102443 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:00.102410    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-27zpv\" (UniqueName: \"kubernetes.io/projected/85974022-4d26-4e54-bda4-3b39cce716cd-kube-api-access-27zpv\") pod \"odh-model-controller-7998c67dc4-qrhf8\" (UID: \"85974022-4d26-4e54-bda4-3b39cce716cd\") " pod="opendatahub/odh-model-controller-7998c67dc4-qrhf8"
Jul 07 11:16:00.123768 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:00.123737    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7769b86749-bljkf"]
Jul 07 11:16:00.126848 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:00.126829    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7769b86749-bljkf"
Jul 07 11:16:00.129729 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:00.129666    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 07 11:16:00.129729 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:00.129674    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 07 11:16:00.129941 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:00.129731    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 07 11:16:00.129941 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:00.129736    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 07 11:16:00.129941 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:00.129761    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-z4bzt\""
Jul 07 11:16:00.129941 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:00.129919    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 07 11:16:00.142650 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:00.142627    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7769b86749-bljkf"]
Jul 07 11:16:00.187715 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:00.187670    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7frfd\" (UniqueName: \"kubernetes.io/projected/b74ff282-2c4a-48e0-a352-fa041935ba7d-kube-api-access-7frfd\") pod \"lws-controller-manager-7769b86749-bljkf\" (UID: \"b74ff282-2c4a-48e0-a352-fa041935ba7d\") " pod="openshift-lws-operator/lws-controller-manager-7769b86749-bljkf"
Jul 07 11:16:00.187915 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:00.187729    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b74ff282-2c4a-48e0-a352-fa041935ba7d-cert\") pod \"lws-controller-manager-7769b86749-bljkf\" (UID: \"b74ff282-2c4a-48e0-a352-fa041935ba7d\") " pod="openshift-lws-operator/lws-controller-manager-7769b86749-bljkf"
Jul 07 11:16:00.187915 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:00.187804    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b74ff282-2c4a-48e0-a352-fa041935ba7d-manager-config\") pod \"lws-controller-manager-7769b86749-bljkf\" (UID: \"b74ff282-2c4a-48e0-a352-fa041935ba7d\") " pod="openshift-lws-operator/lws-controller-manager-7769b86749-bljkf"
Jul 07 11:16:00.187915 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:00.187853    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b74ff282-2c4a-48e0-a352-fa041935ba7d-metrics-cert\") pod \"lws-controller-manager-7769b86749-bljkf\" (UID: \"b74ff282-2c4a-48e0-a352-fa041935ba7d\") " pod="openshift-lws-operator/lws-controller-manager-7769b86749-bljkf"
Jul 07 11:16:00.289047 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:00.289014    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7frfd\" (UniqueName: \"kubernetes.io/projected/b74ff282-2c4a-48e0-a352-fa041935ba7d-kube-api-access-7frfd\") pod \"lws-controller-manager-7769b86749-bljkf\" (UID: \"b74ff282-2c4a-48e0-a352-fa041935ba7d\") " pod="openshift-lws-operator/lws-controller-manager-7769b86749-bljkf"
Jul 07 11:16:00.289253 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:00.289063    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b74ff282-2c4a-48e0-a352-fa041935ba7d-cert\") pod \"lws-controller-manager-7769b86749-bljkf\" (UID: \"b74ff282-2c4a-48e0-a352-fa041935ba7d\") " pod="openshift-lws-operator/lws-controller-manager-7769b86749-bljkf"
Jul 07 11:16:00.289253 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:00.289119    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b74ff282-2c4a-48e0-a352-fa041935ba7d-manager-config\") pod \"lws-controller-manager-7769b86749-bljkf\" (UID: \"b74ff282-2c4a-48e0-a352-fa041935ba7d\") " pod="openshift-lws-operator/lws-controller-manager-7769b86749-bljkf"
Jul 07 11:16:00.289253 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:00.289156    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b74ff282-2c4a-48e0-a352-fa041935ba7d-metrics-cert\") pod \"lws-controller-manager-7769b86749-bljkf\" (UID: \"b74ff282-2c4a-48e0-a352-fa041935ba7d\") " pod="openshift-lws-operator/lws-controller-manager-7769b86749-bljkf"
Jul 07 11:16:00.289837 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:00.289814    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b74ff282-2c4a-48e0-a352-fa041935ba7d-manager-config\") pod \"lws-controller-manager-7769b86749-bljkf\" (UID: \"b74ff282-2c4a-48e0-a352-fa041935ba7d\") " pod="openshift-lws-operator/lws-controller-manager-7769b86749-bljkf"
Jul 07 11:16:00.291680 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:00.291655    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b74ff282-2c4a-48e0-a352-fa041935ba7d-metrics-cert\") pod \"lws-controller-manager-7769b86749-bljkf\" (UID: \"b74ff282-2c4a-48e0-a352-fa041935ba7d\") " pod="openshift-lws-operator/lws-controller-manager-7769b86749-bljkf"
Jul 07 11:16:00.291833 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:00.291812    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b74ff282-2c4a-48e0-a352-fa041935ba7d-cert\") pod \"lws-controller-manager-7769b86749-bljkf\" (UID: \"b74ff282-2c4a-48e0-a352-fa041935ba7d\") " pod="openshift-lws-operator/lws-controller-manager-7769b86749-bljkf"
Jul 07 11:16:00.301461 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:00.301421    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7frfd\" (UniqueName: \"kubernetes.io/projected/b74ff282-2c4a-48e0-a352-fa041935ba7d-kube-api-access-7frfd\") pod \"lws-controller-manager-7769b86749-bljkf\" (UID: \"b74ff282-2c4a-48e0-a352-fa041935ba7d\") " pod="openshift-lws-operator/lws-controller-manager-7769b86749-bljkf"
Jul 07 11:16:00.436394 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:00.436359    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7769b86749-bljkf"
Jul 07 11:16:00.591766 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:00.591677    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/85974022-4d26-4e54-bda4-3b39cce716cd-cert\") pod \"odh-model-controller-7998c67dc4-qrhf8\" (UID: \"85974022-4d26-4e54-bda4-3b39cce716cd\") " pod="opendatahub/odh-model-controller-7998c67dc4-qrhf8"
Jul 07 11:16:00.594206 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:00.594182    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/85974022-4d26-4e54-bda4-3b39cce716cd-cert\") pod \"odh-model-controller-7998c67dc4-qrhf8\" (UID: \"85974022-4d26-4e54-bda4-3b39cce716cd\") " pod="opendatahub/odh-model-controller-7998c67dc4-qrhf8"
Jul 07 11:16:00.603507 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:00.603484    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7769b86749-bljkf"]
Jul 07 11:16:00.604533 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:16:00.604513    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb74ff282_2c4a_48e0_a352_fa041935ba7d.slice/crio-a463a1991e9c305b576053866fa0a373804c6334e7c2383b1d5352756c1a23ae WatchSource:0}: Error finding container a463a1991e9c305b576053866fa0a373804c6334e7c2383b1d5352756c1a23ae: Status 404 returned error can't find the container with id a463a1991e9c305b576053866fa0a373804c6334e7c2383b1d5352756c1a23ae
Jul 07 11:16:00.606500 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:00.606485    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 11:16:00.864222 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:00.864139    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-qrhf8"
Jul 07 11:16:01.000427 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:01.000390    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-qrhf8"]
Jul 07 11:16:01.002765 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:16:01.002742    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85974022_4d26_4e54_bda4_3b39cce716cd.slice/crio-452c11389e2ece0e34b096265b16996e5d02137d53775ab1ab7f4a803cdd0f0c WatchSource:0}: Error finding container 452c11389e2ece0e34b096265b16996e5d02137d53775ab1ab7f4a803cdd0f0c: Status 404 returned error can't find the container with id 452c11389e2ece0e34b096265b16996e5d02137d53775ab1ab7f4a803cdd0f0c
Jul 07 11:16:01.230082 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:01.230045    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-qrhf8" event={"ID":"85974022-4d26-4e54-bda4-3b39cce716cd","Type":"ContainerStarted","Data":"452c11389e2ece0e34b096265b16996e5d02137d53775ab1ab7f4a803cdd0f0c"}
Jul 07 11:16:01.231047 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:01.231028    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7769b86749-bljkf" event={"ID":"b74ff282-2c4a-48e0-a352-fa041935ba7d","Type":"ContainerStarted","Data":"a463a1991e9c305b576053866fa0a373804c6334e7c2383b1d5352756c1a23ae"}
Jul 07 11:16:05.250309 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:05.250268    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-qrhf8" event={"ID":"85974022-4d26-4e54-bda4-3b39cce716cd","Type":"ContainerStarted","Data":"5a7ff990af6e4837d50ff9f2c4948d12c86c9b8ff0d1b351b4ee3646812042b8"}
Jul 07 11:16:05.250778 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:05.250360    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-qrhf8"
Jul 07 11:16:05.251936 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:05.251907    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7769b86749-bljkf" event={"ID":"b74ff282-2c4a-48e0-a352-fa041935ba7d","Type":"ContainerStarted","Data":"fb5f026db967fb017515556e02fa8117b8a6bdbebd785478537e9ab6dc34c629"}
Jul 07 11:16:05.252083 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:05.252067    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7769b86749-bljkf"
Jul 07 11:16:05.337219 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:05.337170    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-qrhf8" podStartSLOduration=2.830705584 podStartE2EDuration="6.337155315s" totalImagesPullingTime="3.506449731s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:15:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:05.336144076 +0000 UTC m=+315.046040462" watchObservedRunningTime="2026-07-07 11:16:05.337155315 +0000 UTC m=+315.047051749"
Jul 07 11:16:05.410770 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:05.410718    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7769b86749-bljkf" podStartSLOduration=1.5492067120000002 podStartE2EDuration="5.41070115s" totalImagesPullingTime="3.861494438s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:16:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:05.409316478 +0000 UTC m=+315.119212865" watchObservedRunningTime="2026-07-07 11:16:05.41070115 +0000 UTC m=+315.120597536"
Jul 07 11:16:11.205771 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:11.205726    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-9k6t8"]
Jul 07 11:16:11.209473 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:11.209454    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-ht2l6"]
Jul 07 11:16:11.209599 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:11.209581    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-9k6t8"
Jul 07 11:16:11.212722 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:11.212697    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ht2l6"
Jul 07 11:16:11.213665 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:11.213646    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 07 11:16:11.214230 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:11.214212    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-vnc9x\""
Jul 07 11:16:11.216164 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:11.216135    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-mbhnp\""
Jul 07 11:16:11.216265 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:11.216183    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 07 11:16:11.220683 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:11.220656    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-9k6t8"]
Jul 07 11:16:11.243987 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:11.243959    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-ht2l6"]
Jul 07 11:16:11.282150 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:11.282119    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nz66\" (UniqueName: \"kubernetes.io/projected/282ba625-e484-45e0-b00a-962c5431ffba-kube-api-access-2nz66\") pod \"llmisvc-controller-manager-777c6bd7d5-ht2l6\" (UID: \"282ba625-e484-45e0-b00a-962c5431ffba\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ht2l6"
Jul 07 11:16:11.282353 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:11.282182    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c4efd1dc-7a63-47d3-a8af-ce5e8e9402e1-cert\") pod \"kserve-controller-manager-d6f4b65b8-9k6t8\" (UID: \"c4efd1dc-7a63-47d3-a8af-ce5e8e9402e1\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-9k6t8"
Jul 07 11:16:11.282353 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:11.282201    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlwnx\" (UniqueName: \"kubernetes.io/projected/c4efd1dc-7a63-47d3-a8af-ce5e8e9402e1-kube-api-access-tlwnx\") pod \"kserve-controller-manager-d6f4b65b8-9k6t8\" (UID: \"c4efd1dc-7a63-47d3-a8af-ce5e8e9402e1\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-9k6t8"
Jul 07 11:16:11.282353 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:11.282236    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/282ba625-e484-45e0-b00a-962c5431ffba-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ht2l6\" (UID: \"282ba625-e484-45e0-b00a-962c5431ffba\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ht2l6"
Jul 07 11:16:11.382570 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:11.382542    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nz66\" (UniqueName: \"kubernetes.io/projected/282ba625-e484-45e0-b00a-962c5431ffba-kube-api-access-2nz66\") pod \"llmisvc-controller-manager-777c6bd7d5-ht2l6\" (UID: \"282ba625-e484-45e0-b00a-962c5431ffba\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ht2l6"
Jul 07 11:16:11.382745 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:11.382592    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c4efd1dc-7a63-47d3-a8af-ce5e8e9402e1-cert\") pod \"kserve-controller-manager-d6f4b65b8-9k6t8\" (UID: \"c4efd1dc-7a63-47d3-a8af-ce5e8e9402e1\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-9k6t8"
Jul 07 11:16:11.382745 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:11.382616    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlwnx\" (UniqueName: \"kubernetes.io/projected/c4efd1dc-7a63-47d3-a8af-ce5e8e9402e1-kube-api-access-tlwnx\") pod \"kserve-controller-manager-d6f4b65b8-9k6t8\" (UID: \"c4efd1dc-7a63-47d3-a8af-ce5e8e9402e1\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-9k6t8"
Jul 07 11:16:11.382745 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:11.382655    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/282ba625-e484-45e0-b00a-962c5431ffba-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ht2l6\" (UID: \"282ba625-e484-45e0-b00a-962c5431ffba\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ht2l6"
Jul 07 11:16:11.382892 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:16:11.382777    2570 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 07 11:16:11.382892 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:16:11.382851    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/282ba625-e484-45e0-b00a-962c5431ffba-cert podName:282ba625-e484-45e0-b00a-962c5431ffba nodeName:}" failed. No retries permitted until 2026-07-07 11:16:11.882830631 +0000 UTC m=+321.592727003 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/282ba625-e484-45e0-b00a-962c5431ffba-cert") pod "llmisvc-controller-manager-777c6bd7d5-ht2l6" (UID: "282ba625-e484-45e0-b00a-962c5431ffba") : secret "llmisvc-webhook-server-cert" not found
Jul 07 11:16:11.385172 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:11.385151    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c4efd1dc-7a63-47d3-a8af-ce5e8e9402e1-cert\") pod \"kserve-controller-manager-d6f4b65b8-9k6t8\" (UID: \"c4efd1dc-7a63-47d3-a8af-ce5e8e9402e1\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-9k6t8"
Jul 07 11:16:11.403487 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:11.403461    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nz66\" (UniqueName: \"kubernetes.io/projected/282ba625-e484-45e0-b00a-962c5431ffba-kube-api-access-2nz66\") pod \"llmisvc-controller-manager-777c6bd7d5-ht2l6\" (UID: \"282ba625-e484-45e0-b00a-962c5431ffba\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ht2l6"
Jul 07 11:16:11.403598 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:11.403457    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlwnx\" (UniqueName: \"kubernetes.io/projected/c4efd1dc-7a63-47d3-a8af-ce5e8e9402e1-kube-api-access-tlwnx\") pod \"kserve-controller-manager-d6f4b65b8-9k6t8\" (UID: \"c4efd1dc-7a63-47d3-a8af-ce5e8e9402e1\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-9k6t8"
Jul 07 11:16:11.523068 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:11.523036    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-9k6t8"
Jul 07 11:16:11.670468 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:11.670445    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-9k6t8"]
Jul 07 11:16:11.672378 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:16:11.672350    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4efd1dc_7a63_47d3_a8af_ce5e8e9402e1.slice/crio-8bb2bd8302f007360823152ae0db63eed35837cceadde00f398be0196e71b417 WatchSource:0}: Error finding container 8bb2bd8302f007360823152ae0db63eed35837cceadde00f398be0196e71b417: Status 404 returned error can't find the container with id 8bb2bd8302f007360823152ae0db63eed35837cceadde00f398be0196e71b417
Jul 07 11:16:11.888602 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:11.888512    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/282ba625-e484-45e0-b00a-962c5431ffba-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ht2l6\" (UID: \"282ba625-e484-45e0-b00a-962c5431ffba\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ht2l6"
Jul 07 11:16:11.890991 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:11.890971    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/282ba625-e484-45e0-b00a-962c5431ffba-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ht2l6\" (UID: \"282ba625-e484-45e0-b00a-962c5431ffba\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ht2l6"
Jul 07 11:16:12.129661 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:12.129624    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ht2l6"
Jul 07 11:16:12.281657 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:12.281622    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-9k6t8" event={"ID":"c4efd1dc-7a63-47d3-a8af-ce5e8e9402e1","Type":"ContainerStarted","Data":"8bb2bd8302f007360823152ae0db63eed35837cceadde00f398be0196e71b417"}
Jul 07 11:16:12.284875 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:16:12.284851    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod282ba625_e484_45e0_b00a_962c5431ffba.slice/crio-061ecaf3db52f39f8c76becf40483b21766bcc539ddfa06ec2c9d531be09e897 WatchSource:0}: Error finding container 061ecaf3db52f39f8c76becf40483b21766bcc539ddfa06ec2c9d531be09e897: Status 404 returned error can't find the container with id 061ecaf3db52f39f8c76becf40483b21766bcc539ddfa06ec2c9d531be09e897
Jul 07 11:16:12.284996 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:12.284975    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-ht2l6"]
Jul 07 11:16:13.288206 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:13.288169    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ht2l6" event={"ID":"282ba625-e484-45e0-b00a-962c5431ffba","Type":"ContainerStarted","Data":"061ecaf3db52f39f8c76becf40483b21766bcc539ddfa06ec2c9d531be09e897"}
Jul 07 11:16:15.300103 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:15.300064    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-9k6t8" event={"ID":"c4efd1dc-7a63-47d3-a8af-ce5e8e9402e1","Type":"ContainerStarted","Data":"f3354402f70cefd81f00dc4b294457e84337d97451b1b1403f53bba2da7a4421"}
Jul 07 11:16:15.300585 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:15.300196    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-9k6t8"
Jul 07 11:16:15.329112 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:15.329058    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-9k6t8" podStartSLOduration=1.23140768 podStartE2EDuration="4.329041558s" totalImagesPullingTime="3.097633878s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:16:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:15.329025202 +0000 UTC m=+325.038921592" watchObservedRunningTime="2026-07-07 11:16:15.329041558 +0000 UTC m=+325.038937945"
Jul 07 11:16:16.258792 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:16.258765    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7769b86749-bljkf"
Jul 07 11:16:16.258966 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:16.258827    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-qrhf8"
Jul 07 11:16:16.306585 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:16.306549    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ht2l6" event={"ID":"282ba625-e484-45e0-b00a-962c5431ffba","Type":"ContainerStarted","Data":"dd0f6eac0636d93fec577f4aad1032a9dea425eea4535172b80ae8f0cb44d83f"}
Jul 07 11:16:16.324754 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:16.324704    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ht2l6" podStartSLOduration=1.879387467 podStartE2EDuration="5.324688735s" totalImagesPullingTime="3.445301268s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:16:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:16.324593126 +0000 UTC m=+326.034489516" watchObservedRunningTime="2026-07-07 11:16:16.324688735 +0000 UTC m=+326.034585121"
Jul 07 11:16:17.310719 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:17.310688    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ht2l6"
Jul 07 11:16:46.311794 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:46.311721    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-9k6t8"
Jul 07 11:16:48.317508 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:16:48.317479    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ht2l6"
Jul 07 11:17:00.837267 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:17:00.837227    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lccrg"]
Jul 07 11:17:00.842045 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:17:00.842024    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-lccrg"
Jul 07 11:17:00.847085 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:17:00.847059    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 07 11:17:00.847389 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:17:00.847372    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 07 11:17:00.850306 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:17:00.850288    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-hdbmv\""
Jul 07 11:17:00.854423 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:17:00.854400    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lccrg"]
Jul 07 11:17:00.901443 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:17:00.901411    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clbd2\" (UniqueName: \"kubernetes.io/projected/a9b73f5f-f512-4ffd-a7dc-ba6c945d7414-kube-api-access-clbd2\") pod \"kuadrant-operator-catalog-lccrg\" (UID: \"a9b73f5f-f512-4ffd-a7dc-ba6c945d7414\") " pod="kuadrant-system/kuadrant-operator-catalog-lccrg"
Jul 07 11:17:01.002615 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:17:01.002581    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clbd2\" (UniqueName: \"kubernetes.io/projected/a9b73f5f-f512-4ffd-a7dc-ba6c945d7414-kube-api-access-clbd2\") pod \"kuadrant-operator-catalog-lccrg\" (UID: \"a9b73f5f-f512-4ffd-a7dc-ba6c945d7414\") " pod="kuadrant-system/kuadrant-operator-catalog-lccrg"
Jul 07 11:17:01.012143 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:17:01.012118    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-clbd2\" (UniqueName: \"kubernetes.io/projected/a9b73f5f-f512-4ffd-a7dc-ba6c945d7414-kube-api-access-clbd2\") pod \"kuadrant-operator-catalog-lccrg\" (UID: \"a9b73f5f-f512-4ffd-a7dc-ba6c945d7414\") " pod="kuadrant-system/kuadrant-operator-catalog-lccrg"
Jul 07 11:17:01.152726 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:17:01.152638    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-lccrg"
Jul 07 11:17:01.289298 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:17:01.289272    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lccrg"]
Jul 07 11:17:01.291712 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:17:01.291686    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9b73f5f_f512_4ffd_a7dc_ba6c945d7414.slice/crio-d07b5343e2036f052c933ca351db79bbf0482ae27928db20cb84c5e9e9b7c3d0 WatchSource:0}: Error finding container d07b5343e2036f052c933ca351db79bbf0482ae27928db20cb84c5e9e9b7c3d0: Status 404 returned error can't find the container with id d07b5343e2036f052c933ca351db79bbf0482ae27928db20cb84c5e9e9b7c3d0
Jul 07 11:17:01.488392 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:17:01.488359    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-lccrg" event={"ID":"a9b73f5f-f512-4ffd-a7dc-ba6c945d7414","Type":"ContainerStarted","Data":"d07b5343e2036f052c933ca351db79bbf0482ae27928db20cb84c5e9e9b7c3d0"}
Jul 07 11:17:03.500590 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:17:03.500548    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-lccrg" event={"ID":"a9b73f5f-f512-4ffd-a7dc-ba6c945d7414","Type":"ContainerStarted","Data":"3b3369d06cc0b103661ece41f7cfa6d95df7d93c60c8f4f583c2165db07981ae"}
Jul 07 11:17:03.521362 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:17:03.521294    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-lccrg" podStartSLOduration=1.463531814 podStartE2EDuration="3.521276517s" totalImagesPullingTime="2.057744703s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:17:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:17:03.519772055 +0000 UTC m=+373.229668444" watchObservedRunningTime="2026-07-07 11:17:03.521276517 +0000 UTC m=+373.231172904"
Jul 07 11:17:11.153284 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:17:11.153229    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-lccrg"
Jul 07 11:17:11.153735 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:17:11.153317    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-lccrg"
Jul 07 11:17:11.176455 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:17:11.176423    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-lccrg"
Jul 07 11:17:11.554737 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:17:11.554707    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-lccrg"
Jul 07 11:17:35.822197 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:17:35.822157    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-lmghr"]
Jul 07 11:17:35.828958 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:17:35.828940    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lmghr"
Jul 07 11:17:35.839824 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:17:35.839797    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-mg8dk\""
Jul 07 11:17:35.861028 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:17:35.861002    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-lmghr"]
Jul 07 11:17:35.905536 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:17:35.905502    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbsrx\" (UniqueName: \"kubernetes.io/projected/3ea4410a-8584-4082-81a7-44b639de86a7-kube-api-access-bbsrx\") pod \"authorino-operator-75cc8d5cb7-lmghr\" (UID: \"3ea4410a-8584-4082-81a7-44b639de86a7\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-lmghr"
Jul 07 11:17:36.005949 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:17:36.005913    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbsrx\" (UniqueName: \"kubernetes.io/projected/3ea4410a-8584-4082-81a7-44b639de86a7-kube-api-access-bbsrx\") pod \"authorino-operator-75cc8d5cb7-lmghr\" (UID: \"3ea4410a-8584-4082-81a7-44b639de86a7\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-lmghr"
Jul 07 11:17:36.027080 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:17:36.027046    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbsrx\" (UniqueName: \"kubernetes.io/projected/3ea4410a-8584-4082-81a7-44b639de86a7-kube-api-access-bbsrx\") pod \"authorino-operator-75cc8d5cb7-lmghr\" (UID: \"3ea4410a-8584-4082-81a7-44b639de86a7\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-lmghr"
Jul 07 11:17:36.138594 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:17:36.138507    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lmghr"
Jul 07 11:17:36.276873 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:17:36.276842    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-lmghr"]
Jul 07 11:17:36.280035 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:17:36.280002    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ea4410a_8584_4082_81a7_44b639de86a7.slice/crio-a526afbc533c1d76ccea440b420fa6d975492202d5337666d2308e378df19b44 WatchSource:0}: Error finding container a526afbc533c1d76ccea440b420fa6d975492202d5337666d2308e378df19b44: Status 404 returned error can't find the container with id a526afbc533c1d76ccea440b420fa6d975492202d5337666d2308e378df19b44
Jul 07 11:17:36.638825 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:17:36.638779    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lmghr" event={"ID":"3ea4410a-8584-4082-81a7-44b639de86a7","Type":"ContainerStarted","Data":"a526afbc533c1d76ccea440b420fa6d975492202d5337666d2308e378df19b44"}
Jul 07 11:17:38.650044 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:17:38.650009    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lmghr" event={"ID":"3ea4410a-8584-4082-81a7-44b639de86a7","Type":"ContainerStarted","Data":"1480d0ef328f5ce24fa426ab298c14ae4ee5247963bbf669869de67cb3684055"}
Jul 07 11:17:38.650485 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:17:38.650149    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lmghr"
Jul 07 11:17:38.689272 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:17:38.689218    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lmghr" podStartSLOduration=1.484700202 podStartE2EDuration="3.689202053s" totalImagesPullingTime="2.204501851s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:17:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:17:38.687871245 +0000 UTC m=+408.397767633" watchObservedRunningTime="2026-07-07 11:17:38.689202053 +0000 UTC m=+408.399098462"
Jul 07 11:17:49.661861 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:17:49.661830    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lmghr"
Jul 07 11:18:05.856086 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:18:05.856053    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-crrvk"]
Jul 07 11:18:05.859501 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:18:05.859484    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-crrvk"
Jul 07 11:18:05.864906 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:18:05.864886    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-6tc45\""
Jul 07 11:18:05.882418 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:18:05.882391    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-crrvk"]
Jul 07 11:18:05.956676 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:18:05.956641    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r89wk\" (UniqueName: \"kubernetes.io/projected/a8053105-3f63-440e-bc40-528c2c56344c-kube-api-access-r89wk\") pod \"limitador-operator-controller-manager-859d7b78bd-crrvk\" (UID: \"a8053105-3f63-440e-bc40-528c2c56344c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-crrvk"
Jul 07 11:18:06.057441 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:18:06.057399    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r89wk\" (UniqueName: \"kubernetes.io/projected/a8053105-3f63-440e-bc40-528c2c56344c-kube-api-access-r89wk\") pod \"limitador-operator-controller-manager-859d7b78bd-crrvk\" (UID: \"a8053105-3f63-440e-bc40-528c2c56344c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-crrvk"
Jul 07 11:18:06.071054 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:18:06.071027    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r89wk\" (UniqueName: \"kubernetes.io/projected/a8053105-3f63-440e-bc40-528c2c56344c-kube-api-access-r89wk\") pod \"limitador-operator-controller-manager-859d7b78bd-crrvk\" (UID: \"a8053105-3f63-440e-bc40-528c2c56344c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-crrvk"
Jul 07 11:18:06.169181 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:18:06.169089    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-crrvk"
Jul 07 11:18:06.309475 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:18:06.309448    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-crrvk"]
Jul 07 11:18:06.312433 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:18:06.312409    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8053105_3f63_440e_bc40_528c2c56344c.slice/crio-3ecc90ae48778aa2b565f770790dd4b6e2cca64676d74289ef789fe948f84831 WatchSource:0}: Error finding container 3ecc90ae48778aa2b565f770790dd4b6e2cca64676d74289ef789fe948f84831: Status 404 returned error can't find the container with id 3ecc90ae48778aa2b565f770790dd4b6e2cca64676d74289ef789fe948f84831
Jul 07 11:18:06.774864 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:18:06.774824    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-crrvk" event={"ID":"a8053105-3f63-440e-bc40-528c2c56344c","Type":"ContainerStarted","Data":"3ecc90ae48778aa2b565f770790dd4b6e2cca64676d74289ef789fe948f84831"}
Jul 07 11:18:07.780749 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:18:07.780714    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-crrvk" event={"ID":"a8053105-3f63-440e-bc40-528c2c56344c","Type":"ContainerStarted","Data":"cb9c478cfe9a63763c0057eddf66c9caa64693377601292a09c0af21be40b20b"}
Jul 07 11:18:07.781140 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:18:07.780779    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-crrvk"
Jul 07 11:18:07.803533 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:18:07.803482    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-crrvk" podStartSLOduration=1.456819808 podStartE2EDuration="2.803467523s" totalImagesPullingTime="1.346647715s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:18:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:18:07.802802422 +0000 UTC m=+437.512698813" watchObservedRunningTime="2026-07-07 11:18:07.803467523 +0000 UTC m=+437.513363910"
Jul 07 11:18:18.787479 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:18:18.787404    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-crrvk"
Jul 07 11:18:38.263577 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:18:38.263542    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-vb7hd"]
Jul 07 11:18:38.267083 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:18:38.267064    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-vb7hd"
Jul 07 11:18:38.270419 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:18:38.270398    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-6v8rx\""
Jul 07 11:18:38.279766 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:18:38.279745    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-vb7hd"]
Jul 07 11:18:38.442869 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:18:38.442832    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56lh6\" (UniqueName: \"kubernetes.io/projected/f5815ee9-b2f4-44d1-a7d7-c8ca7c2e5c2c-kube-api-access-56lh6\") pod \"authorino-75d556bd77-vb7hd\" (UID: \"f5815ee9-b2f4-44d1-a7d7-c8ca7c2e5c2c\") " pod="kuadrant-system/authorino-75d556bd77-vb7hd"
Jul 07 11:18:38.544007 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:18:38.543911    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56lh6\" (UniqueName: \"kubernetes.io/projected/f5815ee9-b2f4-44d1-a7d7-c8ca7c2e5c2c-kube-api-access-56lh6\") pod \"authorino-75d556bd77-vb7hd\" (UID: \"f5815ee9-b2f4-44d1-a7d7-c8ca7c2e5c2c\") " pod="kuadrant-system/authorino-75d556bd77-vb7hd"
Jul 07 11:18:38.554669 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:18:38.554638    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-56lh6\" (UniqueName: \"kubernetes.io/projected/f5815ee9-b2f4-44d1-a7d7-c8ca7c2e5c2c-kube-api-access-56lh6\") pod \"authorino-75d556bd77-vb7hd\" (UID: \"f5815ee9-b2f4-44d1-a7d7-c8ca7c2e5c2c\") " pod="kuadrant-system/authorino-75d556bd77-vb7hd"
Jul 07 11:18:38.576029 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:18:38.575992    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-vb7hd"
Jul 07 11:18:38.705084 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:18:38.705050    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-vb7hd"]
Jul 07 11:18:38.709063 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:18:38.709031    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf5815ee9_b2f4_44d1_a7d7_c8ca7c2e5c2c.slice/crio-b004bbe8572dd91566f384c9f6a7732b6e465354987f8e13e71bebff01836003 WatchSource:0}: Error finding container b004bbe8572dd91566f384c9f6a7732b6e465354987f8e13e71bebff01836003: Status 404 returned error can't find the container with id b004bbe8572dd91566f384c9f6a7732b6e465354987f8e13e71bebff01836003
Jul 07 11:18:38.916031 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:18:38.915935    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-vb7hd" event={"ID":"f5815ee9-b2f4-44d1-a7d7-c8ca7c2e5c2c","Type":"ContainerStarted","Data":"b004bbe8572dd91566f384c9f6a7732b6e465354987f8e13e71bebff01836003"}
Jul 07 11:18:41.934804 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:18:41.934752    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-vb7hd" event={"ID":"f5815ee9-b2f4-44d1-a7d7-c8ca7c2e5c2c","Type":"ContainerStarted","Data":"95bdaf167429d38a2ecb5dc3e792c4fdae00f0a0fff37ecd6e5345068e4323a6"}
Jul 07 11:18:41.954967 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:18:41.954902    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-vb7hd" podStartSLOduration=0.841307772 podStartE2EDuration="3.954880289s" totalImagesPullingTime="3.113572517s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:18:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:18:41.951905548 +0000 UTC m=+471.661801938" watchObservedRunningTime="2026-07-07 11:18:41.954880289 +0000 UTC m=+471.664776676"
Jul 07 11:20:11.516583 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:11.516540    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-nskst"]
Jul 07 11:20:11.520046 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:11.520024    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-nskst"
Jul 07 11:20:11.530068 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:11.530041    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-nskst"]
Jul 07 11:20:11.626368 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:11.626316    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2m2vt\" (UniqueName: \"kubernetes.io/projected/45073bfd-82fd-4ccc-ac64-8003533e7c91-kube-api-access-2m2vt\") pod \"authorino-5d4b67c446-nskst\" (UID: \"45073bfd-82fd-4ccc-ac64-8003533e7c91\") " pod="kuadrant-system/authorino-5d4b67c446-nskst"
Jul 07 11:20:11.727393 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:11.727353    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2m2vt\" (UniqueName: \"kubernetes.io/projected/45073bfd-82fd-4ccc-ac64-8003533e7c91-kube-api-access-2m2vt\") pod \"authorino-5d4b67c446-nskst\" (UID: \"45073bfd-82fd-4ccc-ac64-8003533e7c91\") " pod="kuadrant-system/authorino-5d4b67c446-nskst"
Jul 07 11:20:11.739730 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:11.739703    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2m2vt\" (UniqueName: \"kubernetes.io/projected/45073bfd-82fd-4ccc-ac64-8003533e7c91-kube-api-access-2m2vt\") pod \"authorino-5d4b67c446-nskst\" (UID: \"45073bfd-82fd-4ccc-ac64-8003533e7c91\") " pod="kuadrant-system/authorino-5d4b67c446-nskst"
Jul 07 11:20:11.773789 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:11.773719    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-nskst"]
Jul 07 11:20:11.773967 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:11.773954    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-nskst"
Jul 07 11:20:11.811305 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:11.811275    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-68c776f7cc-bdmqj"]
Jul 07 11:20:11.815075 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:11.815036    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-68c776f7cc-bdmqj"
Jul 07 11:20:11.823592 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:11.823546    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-68c776f7cc-bdmqj"]
Jul 07 11:20:11.916875 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:11.916839    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-nskst"]
Jul 07 11:20:11.920504 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:20:11.920475    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod45073bfd_82fd_4ccc_ac64_8003533e7c91.slice/crio-0862ce594f96e862c6d400f0ec157dfcf90ebcaf200714542a56fc2ade061420 WatchSource:0}: Error finding container 0862ce594f96e862c6d400f0ec157dfcf90ebcaf200714542a56fc2ade061420: Status 404 returned error can't find the container with id 0862ce594f96e862c6d400f0ec157dfcf90ebcaf200714542a56fc2ade061420
Jul 07 11:20:11.929341 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:11.929303    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6csvk\" (UniqueName: \"kubernetes.io/projected/39082ad3-e6d4-4bf6-995d-b7f4dfac1913-kube-api-access-6csvk\") pod \"authorino-68c776f7cc-bdmqj\" (UID: \"39082ad3-e6d4-4bf6-995d-b7f4dfac1913\") " pod="kuadrant-system/authorino-68c776f7cc-bdmqj"
Jul 07 11:20:12.030599 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:12.030509    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6csvk\" (UniqueName: \"kubernetes.io/projected/39082ad3-e6d4-4bf6-995d-b7f4dfac1913-kube-api-access-6csvk\") pod \"authorino-68c776f7cc-bdmqj\" (UID: \"39082ad3-e6d4-4bf6-995d-b7f4dfac1913\") " pod="kuadrant-system/authorino-68c776f7cc-bdmqj"
Jul 07 11:20:12.040456 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:12.040420    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6csvk\" (UniqueName: \"kubernetes.io/projected/39082ad3-e6d4-4bf6-995d-b7f4dfac1913-kube-api-access-6csvk\") pod \"authorino-68c776f7cc-bdmqj\" (UID: \"39082ad3-e6d4-4bf6-995d-b7f4dfac1913\") " pod="kuadrant-system/authorino-68c776f7cc-bdmqj"
Jul 07 11:20:12.126421 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:12.126383    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-68c776f7cc-bdmqj"
Jul 07 11:20:12.143496 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:12.143469    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-68c776f7cc-bdmqj"]
Jul 07 11:20:12.263826 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:12.263758    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-68c776f7cc-bdmqj"]
Jul 07 11:20:12.266312 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:20:12.266285    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod39082ad3_e6d4_4bf6_995d_b7f4dfac1913.slice/crio-939edf6399629e42fe28d2897315dc77039f8dfbe999715f9a3cd86c23ce3c82 WatchSource:0}: Error finding container 939edf6399629e42fe28d2897315dc77039f8dfbe999715f9a3cd86c23ce3c82: Status 404 returned error can't find the container with id 939edf6399629e42fe28d2897315dc77039f8dfbe999715f9a3cd86c23ce3c82
Jul 07 11:20:12.330349 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:12.330248    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-68c776f7cc-bdmqj" event={"ID":"39082ad3-e6d4-4bf6-995d-b7f4dfac1913","Type":"ContainerStarted","Data":"939edf6399629e42fe28d2897315dc77039f8dfbe999715f9a3cd86c23ce3c82"}
Jul 07 11:20:12.331515 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:12.331482    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-nskst" event={"ID":"45073bfd-82fd-4ccc-ac64-8003533e7c91","Type":"ContainerStarted","Data":"0862ce594f96e862c6d400f0ec157dfcf90ebcaf200714542a56fc2ade061420"}
Jul 07 11:20:13.337776 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:13.337734    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-68c776f7cc-bdmqj" event={"ID":"39082ad3-e6d4-4bf6-995d-b7f4dfac1913","Type":"ContainerStarted","Data":"48da582de2e85c9cfc99947dd9c94412b5abda55edd799e2f44f32174fb91c45"}
Jul 07 11:20:13.338255 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:13.337758    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-68c776f7cc-bdmqj" podUID="39082ad3-e6d4-4bf6-995d-b7f4dfac1913" containerName="authorino" containerID="cri-o://48da582de2e85c9cfc99947dd9c94412b5abda55edd799e2f44f32174fb91c45" gracePeriod=30
Jul 07 11:20:13.339395 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:13.339371    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-nskst" event={"ID":"45073bfd-82fd-4ccc-ac64-8003533e7c91","Type":"ContainerStarted","Data":"aeafc1d0e2f7d74f3261325cec7b8026b1ab4ca8b1a6da368bf029e3809c2244"}
Jul 07 11:20:13.339497 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:13.339406    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-nskst" podUID="45073bfd-82fd-4ccc-ac64-8003533e7c91" containerName="authorino" containerID="cri-o://aeafc1d0e2f7d74f3261325cec7b8026b1ab4ca8b1a6da368bf029e3809c2244" gracePeriod=30
Jul 07 11:20:13.354920 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:13.354879    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-68c776f7cc-bdmqj" podStartSLOduration=1.9420491530000001 podStartE2EDuration="2.354864344s" totalImagesPullingTime="412.815191ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:20:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:20:13.354015231 +0000 UTC m=+563.063911639" watchObservedRunningTime="2026-07-07 11:20:13.354864344 +0000 UTC m=+563.064760732"
Jul 07 11:20:13.374152 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:13.374100    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-nskst" podStartSLOduration=1.858849979 podStartE2EDuration="2.374083888s" totalImagesPullingTime="515.233909ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:20:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:20:13.372867909 +0000 UTC m=+563.082764295" watchObservedRunningTime="2026-07-07 11:20:13.374083888 +0000 UTC m=+563.083980274"
Jul 07 11:20:13.593833 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:13.593761    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-vb7hd"]
Jul 07 11:20:13.593990 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:13.593967    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-vb7hd" podUID="f5815ee9-b2f4-44d1-a7d7-c8ca7c2e5c2c" containerName="authorino" containerID="cri-o://95bdaf167429d38a2ecb5dc3e792c4fdae00f0a0fff37ecd6e5345068e4323a6" gracePeriod=30
Jul 07 11:20:13.617123 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:13.617096    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-nskst"
Jul 07 11:20:13.657211 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:13.657185    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-68c776f7cc-bdmqj"
Jul 07 11:20:13.747183 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:13.747155    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6csvk\" (UniqueName: \"kubernetes.io/projected/39082ad3-e6d4-4bf6-995d-b7f4dfac1913-kube-api-access-6csvk\") pod \"39082ad3-e6d4-4bf6-995d-b7f4dfac1913\" (UID: \"39082ad3-e6d4-4bf6-995d-b7f4dfac1913\") "
Jul 07 11:20:13.747314 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:13.747227    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2m2vt\" (UniqueName: \"kubernetes.io/projected/45073bfd-82fd-4ccc-ac64-8003533e7c91-kube-api-access-2m2vt\") pod \"45073bfd-82fd-4ccc-ac64-8003533e7c91\" (UID: \"45073bfd-82fd-4ccc-ac64-8003533e7c91\") "
Jul 07 11:20:13.749517 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:13.749481    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45073bfd-82fd-4ccc-ac64-8003533e7c91-kube-api-access-2m2vt" (OuterVolumeSpecName: "kube-api-access-2m2vt") pod "45073bfd-82fd-4ccc-ac64-8003533e7c91" (UID: "45073bfd-82fd-4ccc-ac64-8003533e7c91"). InnerVolumeSpecName "kube-api-access-2m2vt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:20:13.749639 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:13.749570    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39082ad3-e6d4-4bf6-995d-b7f4dfac1913-kube-api-access-6csvk" (OuterVolumeSpecName: "kube-api-access-6csvk") pod "39082ad3-e6d4-4bf6-995d-b7f4dfac1913" (UID: "39082ad3-e6d4-4bf6-995d-b7f4dfac1913"). InnerVolumeSpecName "kube-api-access-6csvk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:20:13.847918 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:13.847848    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-vb7hd"
Jul 07 11:20:13.848125 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:13.848106    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6csvk\" (UniqueName: \"kubernetes.io/projected/39082ad3-e6d4-4bf6-995d-b7f4dfac1913-kube-api-access-6csvk\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:20:13.848175 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:13.848133    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2m2vt\" (UniqueName: \"kubernetes.io/projected/45073bfd-82fd-4ccc-ac64-8003533e7c91-kube-api-access-2m2vt\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:20:13.948736 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:13.948694    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-56lh6\" (UniqueName: \"kubernetes.io/projected/f5815ee9-b2f4-44d1-a7d7-c8ca7c2e5c2c-kube-api-access-56lh6\") pod \"f5815ee9-b2f4-44d1-a7d7-c8ca7c2e5c2c\" (UID: \"f5815ee9-b2f4-44d1-a7d7-c8ca7c2e5c2c\") "
Jul 07 11:20:13.950840 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:13.950819    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5815ee9-b2f4-44d1-a7d7-c8ca7c2e5c2c-kube-api-access-56lh6" (OuterVolumeSpecName: "kube-api-access-56lh6") pod "f5815ee9-b2f4-44d1-a7d7-c8ca7c2e5c2c" (UID: "f5815ee9-b2f4-44d1-a7d7-c8ca7c2e5c2c"). InnerVolumeSpecName "kube-api-access-56lh6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:20:14.049584 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:14.049548    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-56lh6\" (UniqueName: \"kubernetes.io/projected/f5815ee9-b2f4-44d1-a7d7-c8ca7c2e5c2c-kube-api-access-56lh6\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:20:14.344974 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:14.344935    2570 generic.go:356] "Generic (PLEG): container finished" podID="39082ad3-e6d4-4bf6-995d-b7f4dfac1913" containerID="48da582de2e85c9cfc99947dd9c94412b5abda55edd799e2f44f32174fb91c45" exitCode=0
Jul 07 11:20:14.344974 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:14.344978    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-68c776f7cc-bdmqj"
Jul 07 11:20:14.345535 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:14.345027    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-68c776f7cc-bdmqj" event={"ID":"39082ad3-e6d4-4bf6-995d-b7f4dfac1913","Type":"ContainerDied","Data":"48da582de2e85c9cfc99947dd9c94412b5abda55edd799e2f44f32174fb91c45"}
Jul 07 11:20:14.345535 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:14.345070    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-68c776f7cc-bdmqj" event={"ID":"39082ad3-e6d4-4bf6-995d-b7f4dfac1913","Type":"ContainerDied","Data":"939edf6399629e42fe28d2897315dc77039f8dfbe999715f9a3cd86c23ce3c82"}
Jul 07 11:20:14.345535 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:14.345087    2570 scope.go:117] "RemoveContainer" containerID="48da582de2e85c9cfc99947dd9c94412b5abda55edd799e2f44f32174fb91c45"
Jul 07 11:20:14.346401 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:14.346381    2570 generic.go:356] "Generic (PLEG): container finished" podID="45073bfd-82fd-4ccc-ac64-8003533e7c91" containerID="aeafc1d0e2f7d74f3261325cec7b8026b1ab4ca8b1a6da368bf029e3809c2244" exitCode=0
Jul 07 11:20:14.346473 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:14.346429    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-nskst"
Jul 07 11:20:14.346724 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:14.346476    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-nskst" event={"ID":"45073bfd-82fd-4ccc-ac64-8003533e7c91","Type":"ContainerDied","Data":"aeafc1d0e2f7d74f3261325cec7b8026b1ab4ca8b1a6da368bf029e3809c2244"}
Jul 07 11:20:14.346724 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:14.346513    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-nskst" event={"ID":"45073bfd-82fd-4ccc-ac64-8003533e7c91","Type":"ContainerDied","Data":"0862ce594f96e862c6d400f0ec157dfcf90ebcaf200714542a56fc2ade061420"}
Jul 07 11:20:14.348166 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:14.348146    2570 generic.go:356] "Generic (PLEG): container finished" podID="f5815ee9-b2f4-44d1-a7d7-c8ca7c2e5c2c" containerID="95bdaf167429d38a2ecb5dc3e792c4fdae00f0a0fff37ecd6e5345068e4323a6" exitCode=0
Jul 07 11:20:14.348267 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:14.348184    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-vb7hd" event={"ID":"f5815ee9-b2f4-44d1-a7d7-c8ca7c2e5c2c","Type":"ContainerDied","Data":"95bdaf167429d38a2ecb5dc3e792c4fdae00f0a0fff37ecd6e5345068e4323a6"}
Jul 07 11:20:14.348267 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:14.348191    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-vb7hd"
Jul 07 11:20:14.348267 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:14.348202    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-vb7hd" event={"ID":"f5815ee9-b2f4-44d1-a7d7-c8ca7c2e5c2c","Type":"ContainerDied","Data":"b004bbe8572dd91566f384c9f6a7732b6e465354987f8e13e71bebff01836003"}
Jul 07 11:20:14.357094 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:14.357076    2570 scope.go:117] "RemoveContainer" containerID="48da582de2e85c9cfc99947dd9c94412b5abda55edd799e2f44f32174fb91c45"
Jul 07 11:20:14.357422 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:20:14.357396    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48da582de2e85c9cfc99947dd9c94412b5abda55edd799e2f44f32174fb91c45\": container with ID starting with 48da582de2e85c9cfc99947dd9c94412b5abda55edd799e2f44f32174fb91c45 not found: ID does not exist" containerID="48da582de2e85c9cfc99947dd9c94412b5abda55edd799e2f44f32174fb91c45"
Jul 07 11:20:14.357518 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:14.357435    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48da582de2e85c9cfc99947dd9c94412b5abda55edd799e2f44f32174fb91c45"} err="failed to get container status \"48da582de2e85c9cfc99947dd9c94412b5abda55edd799e2f44f32174fb91c45\": rpc error: code = NotFound desc = could not find container \"48da582de2e85c9cfc99947dd9c94412b5abda55edd799e2f44f32174fb91c45\": container with ID starting with 48da582de2e85c9cfc99947dd9c94412b5abda55edd799e2f44f32174fb91c45 not found: ID does not exist"
Jul 07 11:20:14.357518 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:14.357456    2570 scope.go:117] "RemoveContainer" containerID="aeafc1d0e2f7d74f3261325cec7b8026b1ab4ca8b1a6da368bf029e3809c2244"
Jul 07 11:20:14.365982 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:14.365963    2570 scope.go:117] "RemoveContainer" containerID="aeafc1d0e2f7d74f3261325cec7b8026b1ab4ca8b1a6da368bf029e3809c2244"
Jul 07 11:20:14.366211 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:20:14.366194    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aeafc1d0e2f7d74f3261325cec7b8026b1ab4ca8b1a6da368bf029e3809c2244\": container with ID starting with aeafc1d0e2f7d74f3261325cec7b8026b1ab4ca8b1a6da368bf029e3809c2244 not found: ID does not exist" containerID="aeafc1d0e2f7d74f3261325cec7b8026b1ab4ca8b1a6da368bf029e3809c2244"
Jul 07 11:20:14.366253 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:14.366219    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aeafc1d0e2f7d74f3261325cec7b8026b1ab4ca8b1a6da368bf029e3809c2244"} err="failed to get container status \"aeafc1d0e2f7d74f3261325cec7b8026b1ab4ca8b1a6da368bf029e3809c2244\": rpc error: code = NotFound desc = could not find container \"aeafc1d0e2f7d74f3261325cec7b8026b1ab4ca8b1a6da368bf029e3809c2244\": container with ID starting with aeafc1d0e2f7d74f3261325cec7b8026b1ab4ca8b1a6da368bf029e3809c2244 not found: ID does not exist"
Jul 07 11:20:14.366253 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:14.366235    2570 scope.go:117] "RemoveContainer" containerID="95bdaf167429d38a2ecb5dc3e792c4fdae00f0a0fff37ecd6e5345068e4323a6"
Jul 07 11:20:14.373999 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:14.373981    2570 scope.go:117] "RemoveContainer" containerID="95bdaf167429d38a2ecb5dc3e792c4fdae00f0a0fff37ecd6e5345068e4323a6"
Jul 07 11:20:14.374248 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:20:14.374231    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95bdaf167429d38a2ecb5dc3e792c4fdae00f0a0fff37ecd6e5345068e4323a6\": container with ID starting with 95bdaf167429d38a2ecb5dc3e792c4fdae00f0a0fff37ecd6e5345068e4323a6 not found: ID does not exist" containerID="95bdaf167429d38a2ecb5dc3e792c4fdae00f0a0fff37ecd6e5345068e4323a6"
Jul 07 11:20:14.374311 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:14.374254    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95bdaf167429d38a2ecb5dc3e792c4fdae00f0a0fff37ecd6e5345068e4323a6"} err="failed to get container status \"95bdaf167429d38a2ecb5dc3e792c4fdae00f0a0fff37ecd6e5345068e4323a6\": rpc error: code = NotFound desc = could not find container \"95bdaf167429d38a2ecb5dc3e792c4fdae00f0a0fff37ecd6e5345068e4323a6\": container with ID starting with 95bdaf167429d38a2ecb5dc3e792c4fdae00f0a0fff37ecd6e5345068e4323a6 not found: ID does not exist"
Jul 07 11:20:14.399284 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:14.399248    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-nskst"]
Jul 07 11:20:14.406345 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:14.406301    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-nskst"]
Jul 07 11:20:14.426759 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:14.426731    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-vb7hd"]
Jul 07 11:20:14.431763 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:14.431739    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-vb7hd"]
Jul 07 11:20:14.447482 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:14.447458    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-68c776f7cc-bdmqj"]
Jul 07 11:20:14.450478 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:14.450456    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-68c776f7cc-bdmqj"]
Jul 07 11:20:14.820767 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:14.820738    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39082ad3-e6d4-4bf6-995d-b7f4dfac1913" path="/var/lib/kubelet/pods/39082ad3-e6d4-4bf6-995d-b7f4dfac1913/volumes"
Jul 07 11:20:14.821035 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:14.821023    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45073bfd-82fd-4ccc-ac64-8003533e7c91" path="/var/lib/kubelet/pods/45073bfd-82fd-4ccc-ac64-8003533e7c91/volumes"
Jul 07 11:20:14.821288 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:14.821278    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5815ee9-b2f4-44d1-a7d7-c8ca7c2e5c2c" path="/var/lib/kubelet/pods/f5815ee9-b2f4-44d1-a7d7-c8ca7c2e5c2c/volumes"
Jul 07 11:20:18.395658 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:18.395628    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6f9d97df8c-rrxzl"]
Jul 07 11:20:18.396030 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:18.395950    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39082ad3-e6d4-4bf6-995d-b7f4dfac1913" containerName="authorino"
Jul 07 11:20:18.396030 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:18.395961    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="39082ad3-e6d4-4bf6-995d-b7f4dfac1913" containerName="authorino"
Jul 07 11:20:18.396030 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:18.395978    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5815ee9-b2f4-44d1-a7d7-c8ca7c2e5c2c" containerName="authorino"
Jul 07 11:20:18.396030 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:18.395984    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5815ee9-b2f4-44d1-a7d7-c8ca7c2e5c2c" containerName="authorino"
Jul 07 11:20:18.396030 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:18.395995    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45073bfd-82fd-4ccc-ac64-8003533e7c91" containerName="authorino"
Jul 07 11:20:18.396030 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:18.396000    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="45073bfd-82fd-4ccc-ac64-8003533e7c91" containerName="authorino"
Jul 07 11:20:18.396220 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:18.396052    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="39082ad3-e6d4-4bf6-995d-b7f4dfac1913" containerName="authorino"
Jul 07 11:20:18.396220 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:18.396062    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5815ee9-b2f4-44d1-a7d7-c8ca7c2e5c2c" containerName="authorino"
Jul 07 11:20:18.396220 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:18.396069    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="45073bfd-82fd-4ccc-ac64-8003533e7c91" containerName="authorino"
Jul 07 11:20:18.400520 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:18.400504    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f9d97df8c-rrxzl"
Jul 07 11:20:18.404482 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:18.404461    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 07 11:20:18.404613 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:18.404500    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-qlnrl\""
Jul 07 11:20:18.405721 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:18.405707    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 07 11:20:18.412443 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:18.412420    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6f9d97df8c-rrxzl"]
Jul 07 11:20:18.584046 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:18.584010    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/70778685-5e29-4bd7-94df-72ab65a2bc1e-webhook-cert\") pod \"maas-controller-6f9d97df8c-rrxzl\" (UID: \"70778685-5e29-4bd7-94df-72ab65a2bc1e\") " pod="opendatahub/maas-controller-6f9d97df8c-rrxzl"
Jul 07 11:20:18.584275 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:18.584058    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4279\" (UniqueName: \"kubernetes.io/projected/70778685-5e29-4bd7-94df-72ab65a2bc1e-kube-api-access-w4279\") pod \"maas-controller-6f9d97df8c-rrxzl\" (UID: \"70778685-5e29-4bd7-94df-72ab65a2bc1e\") " pod="opendatahub/maas-controller-6f9d97df8c-rrxzl"
Jul 07 11:20:18.608798 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:18.608755    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7bd7d6c999-qfcp9"]
Jul 07 11:20:18.611948 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:18.611932    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bd7d6c999-qfcp9"
Jul 07 11:20:18.626545 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:18.626520    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7bd7d6c999-qfcp9"]
Jul 07 11:20:18.685159 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:18.685069    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4279\" (UniqueName: \"kubernetes.io/projected/70778685-5e29-4bd7-94df-72ab65a2bc1e-kube-api-access-w4279\") pod \"maas-controller-6f9d97df8c-rrxzl\" (UID: \"70778685-5e29-4bd7-94df-72ab65a2bc1e\") " pod="opendatahub/maas-controller-6f9d97df8c-rrxzl"
Jul 07 11:20:18.685349 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:18.685168    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/70778685-5e29-4bd7-94df-72ab65a2bc1e-webhook-cert\") pod \"maas-controller-6f9d97df8c-rrxzl\" (UID: \"70778685-5e29-4bd7-94df-72ab65a2bc1e\") " pod="opendatahub/maas-controller-6f9d97df8c-rrxzl"
Jul 07 11:20:18.685349 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:20:18.685308    2570 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 07 11:20:18.685479 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:20:18.685393    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70778685-5e29-4bd7-94df-72ab65a2bc1e-webhook-cert podName:70778685-5e29-4bd7-94df-72ab65a2bc1e nodeName:}" failed. No retries permitted until 2026-07-07 11:20:19.185373116 +0000 UTC m=+568.895269500 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/70778685-5e29-4bd7-94df-72ab65a2bc1e-webhook-cert") pod "maas-controller-6f9d97df8c-rrxzl" (UID: "70778685-5e29-4bd7-94df-72ab65a2bc1e") : secret "maas-controller-webhook-cert" not found
Jul 07 11:20:18.694767 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:18.694739    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4279\" (UniqueName: \"kubernetes.io/projected/70778685-5e29-4bd7-94df-72ab65a2bc1e-kube-api-access-w4279\") pod \"maas-controller-6f9d97df8c-rrxzl\" (UID: \"70778685-5e29-4bd7-94df-72ab65a2bc1e\") " pod="opendatahub/maas-controller-6f9d97df8c-rrxzl"
Jul 07 11:20:18.786130 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:18.786096    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6c77a500-b555-47d2-a4c4-d5657dac9aad-webhook-cert\") pod \"maas-controller-7bd7d6c999-qfcp9\" (UID: \"6c77a500-b555-47d2-a4c4-d5657dac9aad\") " pod="opendatahub/maas-controller-7bd7d6c999-qfcp9"
Jul 07 11:20:18.786370 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:18.786164    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzbqg\" (UniqueName: \"kubernetes.io/projected/6c77a500-b555-47d2-a4c4-d5657dac9aad-kube-api-access-hzbqg\") pod \"maas-controller-7bd7d6c999-qfcp9\" (UID: \"6c77a500-b555-47d2-a4c4-d5657dac9aad\") " pod="opendatahub/maas-controller-7bd7d6c999-qfcp9"
Jul 07 11:20:18.887270 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:18.887231    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6c77a500-b555-47d2-a4c4-d5657dac9aad-webhook-cert\") pod \"maas-controller-7bd7d6c999-qfcp9\" (UID: \"6c77a500-b555-47d2-a4c4-d5657dac9aad\") " pod="opendatahub/maas-controller-7bd7d6c999-qfcp9"
Jul 07 11:20:18.887489 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:18.887315    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzbqg\" (UniqueName: \"kubernetes.io/projected/6c77a500-b555-47d2-a4c4-d5657dac9aad-kube-api-access-hzbqg\") pod \"maas-controller-7bd7d6c999-qfcp9\" (UID: \"6c77a500-b555-47d2-a4c4-d5657dac9aad\") " pod="opendatahub/maas-controller-7bd7d6c999-qfcp9"
Jul 07 11:20:18.889824 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:18.889800    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6c77a500-b555-47d2-a4c4-d5657dac9aad-webhook-cert\") pod \"maas-controller-7bd7d6c999-qfcp9\" (UID: \"6c77a500-b555-47d2-a4c4-d5657dac9aad\") " pod="opendatahub/maas-controller-7bd7d6c999-qfcp9"
Jul 07 11:20:18.898139 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:18.898113    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzbqg\" (UniqueName: \"kubernetes.io/projected/6c77a500-b555-47d2-a4c4-d5657dac9aad-kube-api-access-hzbqg\") pod \"maas-controller-7bd7d6c999-qfcp9\" (UID: \"6c77a500-b555-47d2-a4c4-d5657dac9aad\") " pod="opendatahub/maas-controller-7bd7d6c999-qfcp9"
Jul 07 11:20:18.922148 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:18.922112    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bd7d6c999-qfcp9"
Jul 07 11:20:19.058928 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:19.058904    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7bd7d6c999-qfcp9"]
Jul 07 11:20:19.061373 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:20:19.061348    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c77a500_b555_47d2_a4c4_d5657dac9aad.slice/crio-6d333adc70df17e1aad299b545c16d6b0a0498b9c193d7f963326c70f65a8da1 WatchSource:0}: Error finding container 6d333adc70df17e1aad299b545c16d6b0a0498b9c193d7f963326c70f65a8da1: Status 404 returned error can't find the container with id 6d333adc70df17e1aad299b545c16d6b0a0498b9c193d7f963326c70f65a8da1
Jul 07 11:20:19.190787 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:19.190750    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/70778685-5e29-4bd7-94df-72ab65a2bc1e-webhook-cert\") pod \"maas-controller-6f9d97df8c-rrxzl\" (UID: \"70778685-5e29-4bd7-94df-72ab65a2bc1e\") " pod="opendatahub/maas-controller-6f9d97df8c-rrxzl"
Jul 07 11:20:19.193291 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:19.193272    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/70778685-5e29-4bd7-94df-72ab65a2bc1e-webhook-cert\") pod \"maas-controller-6f9d97df8c-rrxzl\" (UID: \"70778685-5e29-4bd7-94df-72ab65a2bc1e\") " pod="opendatahub/maas-controller-6f9d97df8c-rrxzl"
Jul 07 11:20:19.310989 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:19.310940    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f9d97df8c-rrxzl"
Jul 07 11:20:19.375363 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:19.375313    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bd7d6c999-qfcp9" event={"ID":"6c77a500-b555-47d2-a4c4-d5657dac9aad","Type":"ContainerStarted","Data":"6d333adc70df17e1aad299b545c16d6b0a0498b9c193d7f963326c70f65a8da1"}
Jul 07 11:20:19.445864 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:19.445833    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6f9d97df8c-rrxzl"]
Jul 07 11:20:19.449594 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:20:19.449566    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70778685_5e29_4bd7_94df_72ab65a2bc1e.slice/crio-25008fa954f8cb3bf5881e11cb1a6692b9df109930e33adfc5e5551a6094b43e WatchSource:0}: Error finding container 25008fa954f8cb3bf5881e11cb1a6692b9df109930e33adfc5e5551a6094b43e: Status 404 returned error can't find the container with id 25008fa954f8cb3bf5881e11cb1a6692b9df109930e33adfc5e5551a6094b43e
Jul 07 11:20:20.384984 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:20.384921    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f9d97df8c-rrxzl" event={"ID":"70778685-5e29-4bd7-94df-72ab65a2bc1e","Type":"ContainerStarted","Data":"25008fa954f8cb3bf5881e11cb1a6692b9df109930e33adfc5e5551a6094b43e"}
Jul 07 11:20:22.397138 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:22.397095    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bd7d6c999-qfcp9" event={"ID":"6c77a500-b555-47d2-a4c4-d5657dac9aad","Type":"ContainerStarted","Data":"0f44ada79754bb6b2078b27cb02e3faafc26d8d1e266687a55c8ad559ff8b5b8"}
Jul 07 11:20:22.397622 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:22.397267    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7bd7d6c999-qfcp9"
Jul 07 11:20:22.398549 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:22.398525    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f9d97df8c-rrxzl" event={"ID":"70778685-5e29-4bd7-94df-72ab65a2bc1e","Type":"ContainerStarted","Data":"f1e25e458e098c23ffa092e86793100f9d8190cf7e1df275a3f1bb75e779a3a8"}
Jul 07 11:20:22.398673 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:22.398658    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6f9d97df8c-rrxzl"
Jul 07 11:20:22.452380 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:22.452309    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7bd7d6c999-qfcp9" podStartSLOduration=1.5498923900000001 podStartE2EDuration="4.452293708s" totalImagesPullingTime="2.902401318s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:20:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:20:22.446308762 +0000 UTC m=+572.156205162" watchObservedRunningTime="2026-07-07 11:20:22.452293708 +0000 UTC m=+572.162190094"
Jul 07 11:20:22.586373 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:22.586304    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6f9d97df8c-rrxzl" podStartSLOduration=2.082987064 podStartE2EDuration="4.586286477s" totalImagesPullingTime="2.503299413s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:20:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:20:22.585231472 +0000 UTC m=+572.295127862" watchObservedRunningTime="2026-07-07 11:20:22.586286477 +0000 UTC m=+572.296182864"
Jul 07 11:20:33.409610 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:33.409579    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6f9d97df8c-rrxzl"
Jul 07 11:20:33.409960 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:33.409645    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7bd7d6c999-qfcp9"
Jul 07 11:20:33.492042 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:33.492010    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6f9d97df8c-rrxzl"]
Jul 07 11:20:33.492288 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:33.492259    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-6f9d97df8c-rrxzl" podUID="70778685-5e29-4bd7-94df-72ab65a2bc1e" containerName="manager" containerID="cri-o://f1e25e458e098c23ffa092e86793100f9d8190cf7e1df275a3f1bb75e779a3a8" gracePeriod=30
Jul 07 11:20:33.734412 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:33.734386    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f9d97df8c-rrxzl"
Jul 07 11:20:33.814030 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:33.813996    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/70778685-5e29-4bd7-94df-72ab65a2bc1e-webhook-cert\") pod \"70778685-5e29-4bd7-94df-72ab65a2bc1e\" (UID: \"70778685-5e29-4bd7-94df-72ab65a2bc1e\") "
Jul 07 11:20:33.814211 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:33.814041    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w4279\" (UniqueName: \"kubernetes.io/projected/70778685-5e29-4bd7-94df-72ab65a2bc1e-kube-api-access-w4279\") pod \"70778685-5e29-4bd7-94df-72ab65a2bc1e\" (UID: \"70778685-5e29-4bd7-94df-72ab65a2bc1e\") "
Jul 07 11:20:33.816494 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:33.816464    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70778685-5e29-4bd7-94df-72ab65a2bc1e-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "70778685-5e29-4bd7-94df-72ab65a2bc1e" (UID: "70778685-5e29-4bd7-94df-72ab65a2bc1e"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 11:20:33.816494 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:33.816484    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70778685-5e29-4bd7-94df-72ab65a2bc1e-kube-api-access-w4279" (OuterVolumeSpecName: "kube-api-access-w4279") pod "70778685-5e29-4bd7-94df-72ab65a2bc1e" (UID: "70778685-5e29-4bd7-94df-72ab65a2bc1e"). InnerVolumeSpecName "kube-api-access-w4279". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:20:33.915476 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:33.915394    2570 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/70778685-5e29-4bd7-94df-72ab65a2bc1e-webhook-cert\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:20:33.915476 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:33.915425    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w4279\" (UniqueName: \"kubernetes.io/projected/70778685-5e29-4bd7-94df-72ab65a2bc1e-kube-api-access-w4279\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:20:34.453353 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:34.453289    2570 generic.go:356] "Generic (PLEG): container finished" podID="70778685-5e29-4bd7-94df-72ab65a2bc1e" containerID="f1e25e458e098c23ffa092e86793100f9d8190cf7e1df275a3f1bb75e779a3a8" exitCode=0
Jul 07 11:20:34.453353 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:34.453355    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f9d97df8c-rrxzl"
Jul 07 11:20:34.453848 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:34.453374    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f9d97df8c-rrxzl" event={"ID":"70778685-5e29-4bd7-94df-72ab65a2bc1e","Type":"ContainerDied","Data":"f1e25e458e098c23ffa092e86793100f9d8190cf7e1df275a3f1bb75e779a3a8"}
Jul 07 11:20:34.453848 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:34.453426    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f9d97df8c-rrxzl" event={"ID":"70778685-5e29-4bd7-94df-72ab65a2bc1e","Type":"ContainerDied","Data":"25008fa954f8cb3bf5881e11cb1a6692b9df109930e33adfc5e5551a6094b43e"}
Jul 07 11:20:34.453848 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:34.453448    2570 scope.go:117] "RemoveContainer" containerID="f1e25e458e098c23ffa092e86793100f9d8190cf7e1df275a3f1bb75e779a3a8"
Jul 07 11:20:34.462750 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:34.462732    2570 scope.go:117] "RemoveContainer" containerID="f1e25e458e098c23ffa092e86793100f9d8190cf7e1df275a3f1bb75e779a3a8"
Jul 07 11:20:34.463018 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:20:34.462999    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1e25e458e098c23ffa092e86793100f9d8190cf7e1df275a3f1bb75e779a3a8\": container with ID starting with f1e25e458e098c23ffa092e86793100f9d8190cf7e1df275a3f1bb75e779a3a8 not found: ID does not exist" containerID="f1e25e458e098c23ffa092e86793100f9d8190cf7e1df275a3f1bb75e779a3a8"
Jul 07 11:20:34.463075 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:34.463032    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1e25e458e098c23ffa092e86793100f9d8190cf7e1df275a3f1bb75e779a3a8"} err="failed to get container status \"f1e25e458e098c23ffa092e86793100f9d8190cf7e1df275a3f1bb75e779a3a8\": rpc error: code = NotFound desc = could not find container \"f1e25e458e098c23ffa092e86793100f9d8190cf7e1df275a3f1bb75e779a3a8\": container with ID starting with f1e25e458e098c23ffa092e86793100f9d8190cf7e1df275a3f1bb75e779a3a8 not found: ID does not exist"
Jul 07 11:20:34.475156 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:34.475132    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6f9d97df8c-rrxzl"]
Jul 07 11:20:34.479978 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:34.479954    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6f9d97df8c-rrxzl"]
Jul 07 11:20:34.820700 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:34.820669    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70778685-5e29-4bd7-94df-72ab65a2bc1e" path="/var/lib/kubelet/pods/70778685-5e29-4bd7-94df-72ab65a2bc1e/volumes"
Jul 07 11:20:50.741621 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:50.741594    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4v7v8_c8c17c90-3e07-4530-8411-d59cef9a1aff/console-operator/2.log"
Jul 07 11:20:50.745365 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:50.745338    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4v7v8_c8c17c90-3e07-4530-8411-d59cef9a1aff/console-operator/2.log"
Jul 07 11:20:55.399439 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:55.399404    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-6c8bcdc88b-7wwlc"]
Jul 07 11:20:55.399814 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:55.399738    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70778685-5e29-4bd7-94df-72ab65a2bc1e" containerName="manager"
Jul 07 11:20:55.399814 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:55.399750    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="70778685-5e29-4bd7-94df-72ab65a2bc1e" containerName="manager"
Jul 07 11:20:55.399814 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:55.399806    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="70778685-5e29-4bd7-94df-72ab65a2bc1e" containerName="manager"
Jul 07 11:20:55.406527 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:55.406511    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-6c8bcdc88b-7wwlc"
Jul 07 11:20:55.409618 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:55.409597    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\""
Jul 07 11:20:55.409744 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:55.409694    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-5dvr6\""
Jul 07 11:20:55.416165 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:55.416130    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-6c8bcdc88b-7wwlc"]
Jul 07 11:20:55.502275 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:55.502244    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71ef84c7-9a12-42d5-b95b-410fe33993ef-maas-api-tls\") pod \"maas-api-6c8bcdc88b-7wwlc\" (UID: \"71ef84c7-9a12-42d5-b95b-410fe33993ef\") " pod="opendatahub/maas-api-6c8bcdc88b-7wwlc"
Jul 07 11:20:55.502471 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:55.502357    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grdq2\" (UniqueName: \"kubernetes.io/projected/71ef84c7-9a12-42d5-b95b-410fe33993ef-kube-api-access-grdq2\") pod \"maas-api-6c8bcdc88b-7wwlc\" (UID: \"71ef84c7-9a12-42d5-b95b-410fe33993ef\") " pod="opendatahub/maas-api-6c8bcdc88b-7wwlc"
Jul 07 11:20:55.502471 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:55.502387    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71ef84c7-9a12-42d5-b95b-410fe33993ef-openshift-service-ca\") pod \"maas-api-6c8bcdc88b-7wwlc\" (UID: \"71ef84c7-9a12-42d5-b95b-410fe33993ef\") " pod="opendatahub/maas-api-6c8bcdc88b-7wwlc"
Jul 07 11:20:55.603488 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:55.603454    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grdq2\" (UniqueName: \"kubernetes.io/projected/71ef84c7-9a12-42d5-b95b-410fe33993ef-kube-api-access-grdq2\") pod \"maas-api-6c8bcdc88b-7wwlc\" (UID: \"71ef84c7-9a12-42d5-b95b-410fe33993ef\") " pod="opendatahub/maas-api-6c8bcdc88b-7wwlc"
Jul 07 11:20:55.603488 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:55.603499    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71ef84c7-9a12-42d5-b95b-410fe33993ef-openshift-service-ca\") pod \"maas-api-6c8bcdc88b-7wwlc\" (UID: \"71ef84c7-9a12-42d5-b95b-410fe33993ef\") " pod="opendatahub/maas-api-6c8bcdc88b-7wwlc"
Jul 07 11:20:55.603755 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:55.603532    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71ef84c7-9a12-42d5-b95b-410fe33993ef-maas-api-tls\") pod \"maas-api-6c8bcdc88b-7wwlc\" (UID: \"71ef84c7-9a12-42d5-b95b-410fe33993ef\") " pod="opendatahub/maas-api-6c8bcdc88b-7wwlc"
Jul 07 11:20:55.604168 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:55.604143    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71ef84c7-9a12-42d5-b95b-410fe33993ef-openshift-service-ca\") pod \"maas-api-6c8bcdc88b-7wwlc\" (UID: \"71ef84c7-9a12-42d5-b95b-410fe33993ef\") " pod="opendatahub/maas-api-6c8bcdc88b-7wwlc"
Jul 07 11:20:55.606067 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:55.606048    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71ef84c7-9a12-42d5-b95b-410fe33993ef-maas-api-tls\") pod \"maas-api-6c8bcdc88b-7wwlc\" (UID: \"71ef84c7-9a12-42d5-b95b-410fe33993ef\") " pod="opendatahub/maas-api-6c8bcdc88b-7wwlc"
Jul 07 11:20:55.614628 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:55.614603    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-grdq2\" (UniqueName: \"kubernetes.io/projected/71ef84c7-9a12-42d5-b95b-410fe33993ef-kube-api-access-grdq2\") pod \"maas-api-6c8bcdc88b-7wwlc\" (UID: \"71ef84c7-9a12-42d5-b95b-410fe33993ef\") " pod="opendatahub/maas-api-6c8bcdc88b-7wwlc"
Jul 07 11:20:55.717756 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:55.717660    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-6c8bcdc88b-7wwlc"
Jul 07 11:20:55.866968 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:55.866945    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-6c8bcdc88b-7wwlc"]
Jul 07 11:20:55.869667 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:20:55.869636    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71ef84c7_9a12_42d5_b95b_410fe33993ef.slice/crio-bf6ea93a2aa9342a36482ee80f41008551a41467d7abd788f30da7eae610489a WatchSource:0}: Error finding container bf6ea93a2aa9342a36482ee80f41008551a41467d7abd788f30da7eae610489a: Status 404 returned error can't find the container with id bf6ea93a2aa9342a36482ee80f41008551a41467d7abd788f30da7eae610489a
Jul 07 11:20:56.399803 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:56.399769    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59f49fd4f6-lrg8c"]
Jul 07 11:20:56.406286 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:56.406256    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59f49fd4f6-lrg8c"
Jul 07 11:20:56.418358 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:56.418339    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 07 11:20:56.429404 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:56.429379    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59f49fd4f6-lrg8c"]
Jul 07 11:20:56.493859 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:56.493825    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-7bbcb654f4-96trx"]
Jul 07 11:20:56.497784 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:56.497767    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7bbcb654f4-96trx"
Jul 07 11:20:56.504805 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:56.504775    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-gzvvp\""
Jul 07 11:20:56.511619 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:56.511595    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/397d2fc4-9780-4e14-960c-15519470ad42-plugins-config-volume\") pod \"payload-pre-processing-59f49fd4f6-lrg8c\" (UID: \"397d2fc4-9780-4e14-960c-15519470ad42\") " pod="openshift-ingress/payload-pre-processing-59f49fd4f6-lrg8c"
Jul 07 11:20:56.511720 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:56.511636    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wx4rp\" (UniqueName: \"kubernetes.io/projected/397d2fc4-9780-4e14-960c-15519470ad42-kube-api-access-wx4rp\") pod \"payload-pre-processing-59f49fd4f6-lrg8c\" (UID: \"397d2fc4-9780-4e14-960c-15519470ad42\") " pod="openshift-ingress/payload-pre-processing-59f49fd4f6-lrg8c"
Jul 07 11:20:56.516599 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:56.516575    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7bbcb654f4-96trx"]
Jul 07 11:20:56.557003 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:56.556949    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-6c8bcdc88b-7wwlc" event={"ID":"71ef84c7-9a12-42d5-b95b-410fe33993ef","Type":"ContainerStarted","Data":"bf6ea93a2aa9342a36482ee80f41008551a41467d7abd788f30da7eae610489a"}
Jul 07 11:20:56.612519 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:56.612478    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/397d2fc4-9780-4e14-960c-15519470ad42-plugins-config-volume\") pod \"payload-pre-processing-59f49fd4f6-lrg8c\" (UID: \"397d2fc4-9780-4e14-960c-15519470ad42\") " pod="openshift-ingress/payload-pre-processing-59f49fd4f6-lrg8c"
Jul 07 11:20:56.612722 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:56.612537    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wx4rp\" (UniqueName: \"kubernetes.io/projected/397d2fc4-9780-4e14-960c-15519470ad42-kube-api-access-wx4rp\") pod \"payload-pre-processing-59f49fd4f6-lrg8c\" (UID: \"397d2fc4-9780-4e14-960c-15519470ad42\") " pod="openshift-ingress/payload-pre-processing-59f49fd4f6-lrg8c"
Jul 07 11:20:56.612722 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:56.612569    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0da2d46b-d50f-44cd-97fa-910881673476-plugins-config-volume\") pod \"payload-processing-7bbcb654f4-96trx\" (UID: \"0da2d46b-d50f-44cd-97fa-910881673476\") " pod="openshift-ingress/payload-processing-7bbcb654f4-96trx"
Jul 07 11:20:56.612722 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:56.612597    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47x7h\" (UniqueName: \"kubernetes.io/projected/0da2d46b-d50f-44cd-97fa-910881673476-kube-api-access-47x7h\") pod \"payload-processing-7bbcb654f4-96trx\" (UID: \"0da2d46b-d50f-44cd-97fa-910881673476\") " pod="openshift-ingress/payload-processing-7bbcb654f4-96trx"
Jul 07 11:20:56.613247 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:56.613221    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/397d2fc4-9780-4e14-960c-15519470ad42-plugins-config-volume\") pod \"payload-pre-processing-59f49fd4f6-lrg8c\" (UID: \"397d2fc4-9780-4e14-960c-15519470ad42\") " pod="openshift-ingress/payload-pre-processing-59f49fd4f6-lrg8c"
Jul 07 11:20:56.627799 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:56.627770    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wx4rp\" (UniqueName: \"kubernetes.io/projected/397d2fc4-9780-4e14-960c-15519470ad42-kube-api-access-wx4rp\") pod \"payload-pre-processing-59f49fd4f6-lrg8c\" (UID: \"397d2fc4-9780-4e14-960c-15519470ad42\") " pod="openshift-ingress/payload-pre-processing-59f49fd4f6-lrg8c"
Jul 07 11:20:56.713702 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:56.713623    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0da2d46b-d50f-44cd-97fa-910881673476-plugins-config-volume\") pod \"payload-processing-7bbcb654f4-96trx\" (UID: \"0da2d46b-d50f-44cd-97fa-910881673476\") " pod="openshift-ingress/payload-processing-7bbcb654f4-96trx"
Jul 07 11:20:56.713702 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:56.713659    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47x7h\" (UniqueName: \"kubernetes.io/projected/0da2d46b-d50f-44cd-97fa-910881673476-kube-api-access-47x7h\") pod \"payload-processing-7bbcb654f4-96trx\" (UID: \"0da2d46b-d50f-44cd-97fa-910881673476\") " pod="openshift-ingress/payload-processing-7bbcb654f4-96trx"
Jul 07 11:20:56.714311 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:56.714290    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0da2d46b-d50f-44cd-97fa-910881673476-plugins-config-volume\") pod \"payload-processing-7bbcb654f4-96trx\" (UID: \"0da2d46b-d50f-44cd-97fa-910881673476\") " pod="openshift-ingress/payload-processing-7bbcb654f4-96trx"
Jul 07 11:20:56.717509 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:56.717490    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59f49fd4f6-lrg8c"
Jul 07 11:20:56.728049 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:56.728021    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-47x7h\" (UniqueName: \"kubernetes.io/projected/0da2d46b-d50f-44cd-97fa-910881673476-kube-api-access-47x7h\") pod \"payload-processing-7bbcb654f4-96trx\" (UID: \"0da2d46b-d50f-44cd-97fa-910881673476\") " pod="openshift-ingress/payload-processing-7bbcb654f4-96trx"
Jul 07 11:20:56.807959 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:56.807928    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7bbcb654f4-96trx"
Jul 07 11:20:56.891459 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:56.891429    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59f49fd4f6-lrg8c"]
Jul 07 11:20:56.894828 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:20:56.894788    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod397d2fc4_9780_4e14_960c_15519470ad42.slice/crio-f841ce774d8bf2043d0c8c0e4449300d136f3b7e540f6d1f1283e9da6e2d2989 WatchSource:0}: Error finding container f841ce774d8bf2043d0c8c0e4449300d136f3b7e540f6d1f1283e9da6e2d2989: Status 404 returned error can't find the container with id f841ce774d8bf2043d0c8c0e4449300d136f3b7e540f6d1f1283e9da6e2d2989
Jul 07 11:20:56.972881 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:56.972859    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7bbcb654f4-96trx"]
Jul 07 11:20:56.974863 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:20:56.974840    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0da2d46b_d50f_44cd_97fa_910881673476.slice/crio-0c9e48db41a427bed33b7518f874c719444d19684f28f150afdc246dfbef048f WatchSource:0}: Error finding container 0c9e48db41a427bed33b7518f874c719444d19684f28f150afdc246dfbef048f: Status 404 returned error can't find the container with id 0c9e48db41a427bed33b7518f874c719444d19684f28f150afdc246dfbef048f
Jul 07 11:20:57.570232 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:57.570145    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7bbcb654f4-96trx" event={"ID":"0da2d46b-d50f-44cd-97fa-910881673476","Type":"ContainerStarted","Data":"0c9e48db41a427bed33b7518f874c719444d19684f28f150afdc246dfbef048f"}
Jul 07 11:20:57.573186 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:57.573155    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59f49fd4f6-lrg8c" event={"ID":"397d2fc4-9780-4e14-960c-15519470ad42","Type":"ContainerStarted","Data":"f841ce774d8bf2043d0c8c0e4449300d136f3b7e540f6d1f1283e9da6e2d2989"}
Jul 07 11:20:59.585887 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:59.585846    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7bbcb654f4-96trx" event={"ID":"0da2d46b-d50f-44cd-97fa-910881673476","Type":"ContainerStarted","Data":"9d880470c71147d7bff8d74e57b1e7bf514a3a38c76a2fbf9b4441ce07eeeaef"}
Jul 07 11:20:59.586312 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:59.585910    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7bbcb654f4-96trx"
Jul 07 11:20:59.587630 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:59.587603    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-6c8bcdc88b-7wwlc" event={"ID":"71ef84c7-9a12-42d5-b95b-410fe33993ef","Type":"ContainerStarted","Data":"fc79e7081f4fc8f87bbfaf1a04402174a0d4dd5140ae5de95f24a76af1bbf25d"}
Jul 07 11:20:59.587751 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:59.587691    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-6c8bcdc88b-7wwlc"
Jul 07 11:20:59.589239 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:59.589216    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59f49fd4f6-lrg8c" event={"ID":"397d2fc4-9780-4e14-960c-15519470ad42","Type":"ContainerStarted","Data":"6f312c889c559c2e669aafd09d593ac5d50f6e301684889f21f1bef743db96ef"}
Jul 07 11:20:59.589382 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:59.589285    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59f49fd4f6-lrg8c"
Jul 07 11:20:59.608874 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:59.608825    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-7bbcb654f4-96trx" podStartSLOduration=1.097450932 podStartE2EDuration="3.608810986s" totalImagesPullingTime="2.511360054s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:20:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:20:59.606449546 +0000 UTC m=+609.316345933" watchObservedRunningTime="2026-07-07 11:20:59.608810986 +0000 UTC m=+609.318707373"
Jul 07 11:20:59.636888 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:59.636836    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-6c8bcdc88b-7wwlc" podStartSLOduration=1.075107123 podStartE2EDuration="4.636820866s" totalImagesPullingTime="3.561713743s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:20:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:20:59.635132815 +0000 UTC m=+609.345029199" watchObservedRunningTime="2026-07-07 11:20:59.636820866 +0000 UTC m=+609.346717253"
Jul 07 11:20:59.663560 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:20:59.663500    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59f49fd4f6-lrg8c" podStartSLOduration=1.066666091 podStartE2EDuration="3.663479355s" totalImagesPullingTime="2.596813264s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:20:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:20:59.660732266 +0000 UTC m=+609.370628663" watchObservedRunningTime="2026-07-07 11:20:59.663479355 +0000 UTC m=+609.373375742"
Jul 07 11:21:05.599872 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:21:05.599839    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-6c8bcdc88b-7wwlc"
Jul 07 11:21:10.596125 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:21:10.596081    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-59f49fd4f6-lrg8c" podUID="397d2fc4-9780-4e14-960c-15519470ad42" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.132.0.35:9004: connect: connection refused"
Jul 07 11:21:10.596575 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:21:10.596497    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7bbcb654f4-96trx"
Jul 07 11:21:20.595146 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:21:20.595061    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59f49fd4f6-lrg8c"
Jul 07 11:21:46.069539 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:21:46.069503    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-d6d849cd9-lwkds"]
Jul 07 11:21:46.072923 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:21:46.072905    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d6d849cd9-lwkds"
Jul 07 11:21:46.076637 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:21:46.076613    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"authorino-oidc-ca\""
Jul 07 11:21:46.077866 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:21:46.077844    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-6v8rx\""
Jul 07 11:21:46.077964 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:21:46.077947    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 07 11:21:46.087905 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:21:46.087882    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-d6d849cd9-lwkds"]
Jul 07 11:21:46.142883 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:21:46.142851    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/a077d512-dd6f-41a4-857a-ec1b83467ebf-oidc-ca\") pod \"authorino-d6d849cd9-lwkds\" (UID: \"a077d512-dd6f-41a4-857a-ec1b83467ebf\") " pod="kuadrant-system/authorino-d6d849cd9-lwkds"
Jul 07 11:21:46.142883 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:21:46.142883    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6rbq\" (UniqueName: \"kubernetes.io/projected/a077d512-dd6f-41a4-857a-ec1b83467ebf-kube-api-access-z6rbq\") pod \"authorino-d6d849cd9-lwkds\" (UID: \"a077d512-dd6f-41a4-857a-ec1b83467ebf\") " pod="kuadrant-system/authorino-d6d849cd9-lwkds"
Jul 07 11:21:46.143099 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:21:46.142910    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a077d512-dd6f-41a4-857a-ec1b83467ebf-tls-cert\") pod \"authorino-d6d849cd9-lwkds\" (UID: \"a077d512-dd6f-41a4-857a-ec1b83467ebf\") " pod="kuadrant-system/authorino-d6d849cd9-lwkds"
Jul 07 11:21:46.244139 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:21:46.244105    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a077d512-dd6f-41a4-857a-ec1b83467ebf-tls-cert\") pod \"authorino-d6d849cd9-lwkds\" (UID: \"a077d512-dd6f-41a4-857a-ec1b83467ebf\") " pod="kuadrant-system/authorino-d6d849cd9-lwkds"
Jul 07 11:21:46.244361 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:21:46.244184    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/a077d512-dd6f-41a4-857a-ec1b83467ebf-oidc-ca\") pod \"authorino-d6d849cd9-lwkds\" (UID: \"a077d512-dd6f-41a4-857a-ec1b83467ebf\") " pod="kuadrant-system/authorino-d6d849cd9-lwkds"
Jul 07 11:21:46.244361 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:21:46.244204    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6rbq\" (UniqueName: \"kubernetes.io/projected/a077d512-dd6f-41a4-857a-ec1b83467ebf-kube-api-access-z6rbq\") pod \"authorino-d6d849cd9-lwkds\" (UID: \"a077d512-dd6f-41a4-857a-ec1b83467ebf\") " pod="kuadrant-system/authorino-d6d849cd9-lwkds"
Jul 07 11:21:46.244876 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:21:46.244847    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/a077d512-dd6f-41a4-857a-ec1b83467ebf-oidc-ca\") pod \"authorino-d6d849cd9-lwkds\" (UID: \"a077d512-dd6f-41a4-857a-ec1b83467ebf\") " pod="kuadrant-system/authorino-d6d849cd9-lwkds"
Jul 07 11:21:46.246711 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:21:46.246689    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a077d512-dd6f-41a4-857a-ec1b83467ebf-tls-cert\") pod \"authorino-d6d849cd9-lwkds\" (UID: \"a077d512-dd6f-41a4-857a-ec1b83467ebf\") " pod="kuadrant-system/authorino-d6d849cd9-lwkds"
Jul 07 11:21:46.255576 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:21:46.255554    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6rbq\" (UniqueName: \"kubernetes.io/projected/a077d512-dd6f-41a4-857a-ec1b83467ebf-kube-api-access-z6rbq\") pod \"authorino-d6d849cd9-lwkds\" (UID: \"a077d512-dd6f-41a4-857a-ec1b83467ebf\") " pod="kuadrant-system/authorino-d6d849cd9-lwkds"
Jul 07 11:21:46.381904 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:21:46.381806    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d6d849cd9-lwkds"
Jul 07 11:21:46.545030 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:21:46.544999    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-d6d849cd9-lwkds"]
Jul 07 11:21:46.546843 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:21:46.546815    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda077d512_dd6f_41a4_857a_ec1b83467ebf.slice/crio-5aa65d23b12bcf51614a0c81622527499f3cd470216f1fd6a4931a47cb03f93c WatchSource:0}: Error finding container 5aa65d23b12bcf51614a0c81622527499f3cd470216f1fd6a4931a47cb03f93c: Status 404 returned error can't find the container with id 5aa65d23b12bcf51614a0c81622527499f3cd470216f1fd6a4931a47cb03f93c
Jul 07 11:21:46.548124 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:21:46.548106    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 11:21:46.800181 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:21:46.800145    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d6d849cd9-lwkds" event={"ID":"a077d512-dd6f-41a4-857a-ec1b83467ebf","Type":"ContainerStarted","Data":"5aa65d23b12bcf51614a0c81622527499f3cd470216f1fd6a4931a47cb03f93c"}
Jul 07 11:21:47.806063 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:21:47.806029    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d6d849cd9-lwkds" event={"ID":"a077d512-dd6f-41a4-857a-ec1b83467ebf","Type":"ContainerStarted","Data":"e68e28ef4d1580c7dc06e074741320c54f7eca1e5c60ec06011fd821bf4458af"}
Jul 07 11:21:47.824539 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:21:47.824492    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-d6d849cd9-lwkds" podStartSLOduration=1.4492130429999999 podStartE2EDuration="1.82447616s" totalImagesPullingTime="375.263117ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:21:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:21:47.823766328 +0000 UTC m=+657.533662707" watchObservedRunningTime="2026-07-07 11:21:47.82447616 +0000 UTC m=+657.534372546"
Jul 07 11:22:34.636048 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:34.636012    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-754fq"]
Jul 07 11:22:34.639560 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:34.639544    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-754fq"
Jul 07 11:22:34.644353 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:34.644308    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 07 11:22:34.644495 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:34.644308    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6wgkw\""
Jul 07 11:22:34.644495 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:34.644309    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 07 11:22:34.644495 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:34.644309    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 07 11:22:34.652105 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:34.652080    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-754fq"]
Jul 07 11:22:34.752704 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:34.752668    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e97b54d-b8dd-40ef-bcf5-4a6ab21e1bbf-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-754fq\" (UID: \"0e97b54d-b8dd-40ef-bcf5-4a6ab21e1bbf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-754fq"
Jul 07 11:22:34.752883 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:34.752715    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e97b54d-b8dd-40ef-bcf5-4a6ab21e1bbf-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-754fq\" (UID: \"0e97b54d-b8dd-40ef-bcf5-4a6ab21e1bbf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-754fq"
Jul 07 11:22:34.752883 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:34.752742    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ws8g\" (UniqueName: \"kubernetes.io/projected/0e97b54d-b8dd-40ef-bcf5-4a6ab21e1bbf-kube-api-access-7ws8g\") pod \"e2e-trlp-test-simulated-kserve-598466548d-754fq\" (UID: \"0e97b54d-b8dd-40ef-bcf5-4a6ab21e1bbf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-754fq"
Jul 07 11:22:34.752883 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:34.752765    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e97b54d-b8dd-40ef-bcf5-4a6ab21e1bbf-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-754fq\" (UID: \"0e97b54d-b8dd-40ef-bcf5-4a6ab21e1bbf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-754fq"
Jul 07 11:22:34.753053 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:34.752882    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e97b54d-b8dd-40ef-bcf5-4a6ab21e1bbf-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-754fq\" (UID: \"0e97b54d-b8dd-40ef-bcf5-4a6ab21e1bbf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-754fq"
Jul 07 11:22:34.753053 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:34.752934    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e97b54d-b8dd-40ef-bcf5-4a6ab21e1bbf-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-754fq\" (UID: \"0e97b54d-b8dd-40ef-bcf5-4a6ab21e1bbf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-754fq"
Jul 07 11:22:34.853559 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:34.853522    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e97b54d-b8dd-40ef-bcf5-4a6ab21e1bbf-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-754fq\" (UID: \"0e97b54d-b8dd-40ef-bcf5-4a6ab21e1bbf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-754fq"
Jul 07 11:22:34.853764 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:34.853602    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e97b54d-b8dd-40ef-bcf5-4a6ab21e1bbf-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-754fq\" (UID: \"0e97b54d-b8dd-40ef-bcf5-4a6ab21e1bbf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-754fq"
Jul 07 11:22:34.853764 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:34.853637    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e97b54d-b8dd-40ef-bcf5-4a6ab21e1bbf-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-754fq\" (UID: \"0e97b54d-b8dd-40ef-bcf5-4a6ab21e1bbf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-754fq"
Jul 07 11:22:34.853764 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:34.853664    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ws8g\" (UniqueName: \"kubernetes.io/projected/0e97b54d-b8dd-40ef-bcf5-4a6ab21e1bbf-kube-api-access-7ws8g\") pod \"e2e-trlp-test-simulated-kserve-598466548d-754fq\" (UID: \"0e97b54d-b8dd-40ef-bcf5-4a6ab21e1bbf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-754fq"
Jul 07 11:22:34.853764 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:34.853690    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e97b54d-b8dd-40ef-bcf5-4a6ab21e1bbf-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-754fq\" (UID: \"0e97b54d-b8dd-40ef-bcf5-4a6ab21e1bbf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-754fq"
Jul 07 11:22:34.853764 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:34.853758    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e97b54d-b8dd-40ef-bcf5-4a6ab21e1bbf-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-754fq\" (UID: \"0e97b54d-b8dd-40ef-bcf5-4a6ab21e1bbf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-754fq"
Jul 07 11:22:34.854023 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:34.854003    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e97b54d-b8dd-40ef-bcf5-4a6ab21e1bbf-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-754fq\" (UID: \"0e97b54d-b8dd-40ef-bcf5-4a6ab21e1bbf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-754fq"
Jul 07 11:22:34.854117 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:34.854092    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e97b54d-b8dd-40ef-bcf5-4a6ab21e1bbf-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-754fq\" (UID: \"0e97b54d-b8dd-40ef-bcf5-4a6ab21e1bbf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-754fq"
Jul 07 11:22:34.854299 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:34.854268    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e97b54d-b8dd-40ef-bcf5-4a6ab21e1bbf-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-754fq\" (UID: \"0e97b54d-b8dd-40ef-bcf5-4a6ab21e1bbf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-754fq"
Jul 07 11:22:34.856917 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:34.856892    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e97b54d-b8dd-40ef-bcf5-4a6ab21e1bbf-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-754fq\" (UID: \"0e97b54d-b8dd-40ef-bcf5-4a6ab21e1bbf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-754fq"
Jul 07 11:22:34.857162 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:34.857139    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e97b54d-b8dd-40ef-bcf5-4a6ab21e1bbf-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-754fq\" (UID: \"0e97b54d-b8dd-40ef-bcf5-4a6ab21e1bbf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-754fq"
Jul 07 11:22:34.863965 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:34.863940    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ws8g\" (UniqueName: \"kubernetes.io/projected/0e97b54d-b8dd-40ef-bcf5-4a6ab21e1bbf-kube-api-access-7ws8g\") pod \"e2e-trlp-test-simulated-kserve-598466548d-754fq\" (UID: \"0e97b54d-b8dd-40ef-bcf5-4a6ab21e1bbf\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-754fq"
Jul 07 11:22:34.949581 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:34.949473    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-754fq"
Jul 07 11:22:35.091004 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:35.090979    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-754fq"]
Jul 07 11:22:35.093123 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:22:35.093096    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e97b54d_b8dd_40ef_bcf5_4a6ab21e1bbf.slice/crio-bf715507fa02244eab05d63f087df09dec7edfb5eb3c1d4f818c8751c0984452 WatchSource:0}: Error finding container bf715507fa02244eab05d63f087df09dec7edfb5eb3c1d4f818c8751c0984452: Status 404 returned error can't find the container with id bf715507fa02244eab05d63f087df09dec7edfb5eb3c1d4f818c8751c0984452
Jul 07 11:22:36.027557 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:36.027480    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-754fq" event={"ID":"0e97b54d-b8dd-40ef-bcf5-4a6ab21e1bbf","Type":"ContainerStarted","Data":"bf715507fa02244eab05d63f087df09dec7edfb5eb3c1d4f818c8751c0984452"}
Jul 07 11:22:37.033218 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:37.033178    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-754fq" event={"ID":"0e97b54d-b8dd-40ef-bcf5-4a6ab21e1bbf","Type":"ContainerStarted","Data":"5728d259caa96dfba65b4b5a9b09a836ebb4966150f91cc849216d3a1f608ce7"}
Jul 07 11:22:37.057973 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:37.057917    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-754fq" podStartSLOduration=1.8374700370000001 podStartE2EDuration="3.057900408s" totalImagesPullingTime="1.220430371s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:22:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:22:37.055386168 +0000 UTC m=+706.765282555" watchObservedRunningTime="2026-07-07 11:22:37.057900408 +0000 UTC m=+706.767796794"
Jul 07 11:22:44.950011 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:44.949922    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-754fq"
Jul 07 11:22:44.950503 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:44.950011    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-754fq"
Jul 07 11:22:44.966636 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:44.966609    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-754fq"
Jul 07 11:22:45.081578 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:45.081550    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-754fq"
Jul 07 11:22:57.518094 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:57.518058    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-jh5w6"]
Jul 07 11:22:57.521687 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:57.521667    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jh5w6"
Jul 07 11:22:57.524914 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:57.524894    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 07 11:22:57.536090 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:57.536065    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-jh5w6"]
Jul 07 11:22:57.652109 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:57.652066    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rp4cr\" (UniqueName: \"kubernetes.io/projected/d8ba409c-d18b-4602-b567-cc0541953a57-kube-api-access-rp4cr\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jh5w6\" (UID: \"d8ba409c-d18b-4602-b567-cc0541953a57\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jh5w6"
Jul 07 11:22:57.652294 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:57.652131    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d8ba409c-d18b-4602-b567-cc0541953a57-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jh5w6\" (UID: \"d8ba409c-d18b-4602-b567-cc0541953a57\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jh5w6"
Jul 07 11:22:57.652294 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:57.652156    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8ba409c-d18b-4602-b567-cc0541953a57-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jh5w6\" (UID: \"d8ba409c-d18b-4602-b567-cc0541953a57\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jh5w6"
Jul 07 11:22:57.652294 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:57.652188    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d8ba409c-d18b-4602-b567-cc0541953a57-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jh5w6\" (UID: \"d8ba409c-d18b-4602-b567-cc0541953a57\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jh5w6"
Jul 07 11:22:57.652294 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:57.652262    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d8ba409c-d18b-4602-b567-cc0541953a57-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jh5w6\" (UID: \"d8ba409c-d18b-4602-b567-cc0541953a57\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jh5w6"
Jul 07 11:22:57.652476 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:57.652313    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d8ba409c-d18b-4602-b567-cc0541953a57-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jh5w6\" (UID: \"d8ba409c-d18b-4602-b567-cc0541953a57\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jh5w6"
Jul 07 11:22:57.753691 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:57.753651    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d8ba409c-d18b-4602-b567-cc0541953a57-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jh5w6\" (UID: \"d8ba409c-d18b-4602-b567-cc0541953a57\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jh5w6"
Jul 07 11:22:57.753882 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:57.753702    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d8ba409c-d18b-4602-b567-cc0541953a57-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jh5w6\" (UID: \"d8ba409c-d18b-4602-b567-cc0541953a57\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jh5w6"
Jul 07 11:22:57.753882 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:57.753729    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d8ba409c-d18b-4602-b567-cc0541953a57-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jh5w6\" (UID: \"d8ba409c-d18b-4602-b567-cc0541953a57\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jh5w6"
Jul 07 11:22:57.753882 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:57.753762    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rp4cr\" (UniqueName: \"kubernetes.io/projected/d8ba409c-d18b-4602-b567-cc0541953a57-kube-api-access-rp4cr\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jh5w6\" (UID: \"d8ba409c-d18b-4602-b567-cc0541953a57\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jh5w6"
Jul 07 11:22:57.753882 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:57.753822    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d8ba409c-d18b-4602-b567-cc0541953a57-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jh5w6\" (UID: \"d8ba409c-d18b-4602-b567-cc0541953a57\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jh5w6"
Jul 07 11:22:57.753882 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:57.753848    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8ba409c-d18b-4602-b567-cc0541953a57-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jh5w6\" (UID: \"d8ba409c-d18b-4602-b567-cc0541953a57\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jh5w6"
Jul 07 11:22:57.754239 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:57.754202    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d8ba409c-d18b-4602-b567-cc0541953a57-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jh5w6\" (UID: \"d8ba409c-d18b-4602-b567-cc0541953a57\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jh5w6"
Jul 07 11:22:57.754239 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:57.754225    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d8ba409c-d18b-4602-b567-cc0541953a57-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jh5w6\" (UID: \"d8ba409c-d18b-4602-b567-cc0541953a57\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jh5w6"
Jul 07 11:22:57.754473 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:57.754287    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8ba409c-d18b-4602-b567-cc0541953a57-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jh5w6\" (UID: \"d8ba409c-d18b-4602-b567-cc0541953a57\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jh5w6"
Jul 07 11:22:57.756364 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:57.756316    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d8ba409c-d18b-4602-b567-cc0541953a57-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jh5w6\" (UID: \"d8ba409c-d18b-4602-b567-cc0541953a57\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jh5w6"
Jul 07 11:22:57.756598 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:57.756579    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d8ba409c-d18b-4602-b567-cc0541953a57-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jh5w6\" (UID: \"d8ba409c-d18b-4602-b567-cc0541953a57\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jh5w6"
Jul 07 11:22:57.764265 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:57.764240    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rp4cr\" (UniqueName: \"kubernetes.io/projected/d8ba409c-d18b-4602-b567-cc0541953a57-kube-api-access-rp4cr\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jh5w6\" (UID: \"d8ba409c-d18b-4602-b567-cc0541953a57\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jh5w6"
Jul 07 11:22:57.832153 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:57.832072    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jh5w6"
Jul 07 11:22:57.974176 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:57.974134    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-jh5w6"]
Jul 07 11:22:57.976463 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:22:57.976436    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8ba409c_d18b_4602_b567_cc0541953a57.slice/crio-69ca4ed6f6f2468597ab53a2887f492f9b9bfe0c20235fc9c6f637ff2eab4d69 WatchSource:0}: Error finding container 69ca4ed6f6f2468597ab53a2887f492f9b9bfe0c20235fc9c6f637ff2eab4d69: Status 404 returned error can't find the container with id 69ca4ed6f6f2468597ab53a2887f492f9b9bfe0c20235fc9c6f637ff2eab4d69
Jul 07 11:22:58.129356 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:58.129240    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jh5w6" event={"ID":"d8ba409c-d18b-4602-b567-cc0541953a57","Type":"ContainerStarted","Data":"69ca4ed6f6f2468597ab53a2887f492f9b9bfe0c20235fc9c6f637ff2eab4d69"}
Jul 07 11:22:59.135067 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:59.135032    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jh5w6" event={"ID":"d8ba409c-d18b-4602-b567-cc0541953a57","Type":"ContainerStarted","Data":"d6260af9f377ce91b52637dee41dd18a231d86f99c4953b11ccff14f766d52ad"}
Jul 07 11:22:59.163739 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:22:59.163683    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jh5w6" podStartSLOduration=1.96964709 podStartE2EDuration="2.163667141s" totalImagesPullingTime="194.020051ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:22:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:22:59.163497987 +0000 UTC m=+728.873394373" watchObservedRunningTime="2026-07-07 11:22:59.163667141 +0000 UTC m=+728.873563543"
Jul 07 11:23:06.264645 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:06.264606    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bmr7n"]
Jul 07 11:23:06.271005 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:06.270980    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bmr7n"
Jul 07 11:23:06.275853 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:06.275827    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 07 11:23:06.296146 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:06.296116    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bmr7n"]
Jul 07 11:23:06.430491 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:06.430456    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d99d53d-53bc-4577-8057-70cf8f53f1e7-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bmr7n\" (UID: \"8d99d53d-53bc-4577-8057-70cf8f53f1e7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bmr7n"
Jul 07 11:23:06.430672 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:06.430529    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d99d53d-53bc-4577-8057-70cf8f53f1e7-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bmr7n\" (UID: \"8d99d53d-53bc-4577-8057-70cf8f53f1e7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bmr7n"
Jul 07 11:23:06.430672 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:06.430557    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvmdq\" (UniqueName: \"kubernetes.io/projected/8d99d53d-53bc-4577-8057-70cf8f53f1e7-kube-api-access-lvmdq\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bmr7n\" (UID: \"8d99d53d-53bc-4577-8057-70cf8f53f1e7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bmr7n"
Jul 07 11:23:06.430672 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:06.430585    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d99d53d-53bc-4577-8057-70cf8f53f1e7-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bmr7n\" (UID: \"8d99d53d-53bc-4577-8057-70cf8f53f1e7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bmr7n"
Jul 07 11:23:06.430672 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:06.430622    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d99d53d-53bc-4577-8057-70cf8f53f1e7-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bmr7n\" (UID: \"8d99d53d-53bc-4577-8057-70cf8f53f1e7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bmr7n"
Jul 07 11:23:06.430835 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:06.430680    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d99d53d-53bc-4577-8057-70cf8f53f1e7-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bmr7n\" (UID: \"8d99d53d-53bc-4577-8057-70cf8f53f1e7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bmr7n"
Jul 07 11:23:06.531838 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:06.531741    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d99d53d-53bc-4577-8057-70cf8f53f1e7-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bmr7n\" (UID: \"8d99d53d-53bc-4577-8057-70cf8f53f1e7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bmr7n"
Jul 07 11:23:06.531838 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:06.531780    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d99d53d-53bc-4577-8057-70cf8f53f1e7-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bmr7n\" (UID: \"8d99d53d-53bc-4577-8057-70cf8f53f1e7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bmr7n"
Jul 07 11:23:06.531838 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:06.531830    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d99d53d-53bc-4577-8057-70cf8f53f1e7-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bmr7n\" (UID: \"8d99d53d-53bc-4577-8057-70cf8f53f1e7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bmr7n"
Jul 07 11:23:06.532118 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:06.531870    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d99d53d-53bc-4577-8057-70cf8f53f1e7-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bmr7n\" (UID: \"8d99d53d-53bc-4577-8057-70cf8f53f1e7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bmr7n"
Jul 07 11:23:06.532118 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:06.531886    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvmdq\" (UniqueName: \"kubernetes.io/projected/8d99d53d-53bc-4577-8057-70cf8f53f1e7-kube-api-access-lvmdq\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bmr7n\" (UID: \"8d99d53d-53bc-4577-8057-70cf8f53f1e7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bmr7n"
Jul 07 11:23:06.532118 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:06.531922    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d99d53d-53bc-4577-8057-70cf8f53f1e7-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bmr7n\" (UID: \"8d99d53d-53bc-4577-8057-70cf8f53f1e7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bmr7n"
Jul 07 11:23:06.532285 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:06.532226    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d99d53d-53bc-4577-8057-70cf8f53f1e7-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bmr7n\" (UID: \"8d99d53d-53bc-4577-8057-70cf8f53f1e7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bmr7n"
Jul 07 11:23:06.532285 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:06.532232    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d99d53d-53bc-4577-8057-70cf8f53f1e7-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bmr7n\" (UID: \"8d99d53d-53bc-4577-8057-70cf8f53f1e7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bmr7n"
Jul 07 11:23:06.532408 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:06.532285    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d99d53d-53bc-4577-8057-70cf8f53f1e7-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bmr7n\" (UID: \"8d99d53d-53bc-4577-8057-70cf8f53f1e7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bmr7n"
Jul 07 11:23:06.534387 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:06.534366    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d99d53d-53bc-4577-8057-70cf8f53f1e7-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bmr7n\" (UID: \"8d99d53d-53bc-4577-8057-70cf8f53f1e7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bmr7n"
Jul 07 11:23:06.534564 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:06.534547    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d99d53d-53bc-4577-8057-70cf8f53f1e7-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bmr7n\" (UID: \"8d99d53d-53bc-4577-8057-70cf8f53f1e7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bmr7n"
Jul 07 11:23:06.541680 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:06.541657    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvmdq\" (UniqueName: \"kubernetes.io/projected/8d99d53d-53bc-4577-8057-70cf8f53f1e7-kube-api-access-lvmdq\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bmr7n\" (UID: \"8d99d53d-53bc-4577-8057-70cf8f53f1e7\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bmr7n"
Jul 07 11:23:06.580606 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:06.580566    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bmr7n"
Jul 07 11:23:06.717790 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:06.717765    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bmr7n"]
Jul 07 11:23:06.719237 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:23:06.719210    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8d99d53d_53bc_4577_8057_70cf8f53f1e7.slice/crio-0bcfbc4c8eb623a4fae134be12be47e79f034d4b1ae5b19ace77b1a95233f085 WatchSource:0}: Error finding container 0bcfbc4c8eb623a4fae134be12be47e79f034d4b1ae5b19ace77b1a95233f085: Status 404 returned error can't find the container with id 0bcfbc4c8eb623a4fae134be12be47e79f034d4b1ae5b19ace77b1a95233f085
Jul 07 11:23:07.178104 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:07.178065    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bmr7n" event={"ID":"8d99d53d-53bc-4577-8057-70cf8f53f1e7","Type":"ContainerStarted","Data":"34386c0dade9e20429756639c1b092d3edf79ade22ad29b6d4dbf54e19f5704c"}
Jul 07 11:23:07.178276 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:07.178114    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bmr7n" event={"ID":"8d99d53d-53bc-4577-8057-70cf8f53f1e7","Type":"ContainerStarted","Data":"0bcfbc4c8eb623a4fae134be12be47e79f034d4b1ae5b19ace77b1a95233f085"}
Jul 07 11:23:07.206547 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:07.206446    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bmr7n" podStartSLOduration=0.994184235 podStartE2EDuration="1.206430199s" totalImagesPullingTime="212.245964ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:23:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:23:07.204529586 +0000 UTC m=+736.914425975" watchObservedRunningTime="2026-07-07 11:23:07.206430199 +0000 UTC m=+736.916326586"
Jul 07 11:23:07.832863 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:07.832821    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jh5w6"
Jul 07 11:23:07.833387 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:07.832881    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jh5w6"
Jul 07 11:23:07.845357 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:07.845308    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jh5w6"
Jul 07 11:23:08.194089 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:08.194014    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jh5w6"
Jul 07 11:23:16.581068 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:16.581020    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bmr7n"
Jul 07 11:23:16.581550 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:16.581115    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bmr7n"
Jul 07 11:23:16.593825 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:16.593799    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bmr7n"
Jul 07 11:23:17.236868 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:17.236836    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bmr7n"
Jul 07 11:23:37.189424 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:37.189386    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-55655b5696-4nnbg"]
Jul 07 11:23:37.192942 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:37.192924    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55655b5696-4nnbg"
Jul 07 11:23:37.211498 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:37.211463    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-55655b5696-4nnbg"]
Jul 07 11:23:37.282465 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:37.282432    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b81abf77-b887-4ef6-9b9d-b6c1f94277db-tls-cert\") pod \"authorino-55655b5696-4nnbg\" (UID: \"b81abf77-b887-4ef6-9b9d-b6c1f94277db\") " pod="kuadrant-system/authorino-55655b5696-4nnbg"
Jul 07 11:23:37.282636 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:37.282475    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/b81abf77-b887-4ef6-9b9d-b6c1f94277db-oidc-ca\") pod \"authorino-55655b5696-4nnbg\" (UID: \"b81abf77-b887-4ef6-9b9d-b6c1f94277db\") " pod="kuadrant-system/authorino-55655b5696-4nnbg"
Jul 07 11:23:37.282636 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:37.282552    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4sz8\" (UniqueName: \"kubernetes.io/projected/b81abf77-b887-4ef6-9b9d-b6c1f94277db-kube-api-access-x4sz8\") pod \"authorino-55655b5696-4nnbg\" (UID: \"b81abf77-b887-4ef6-9b9d-b6c1f94277db\") " pod="kuadrant-system/authorino-55655b5696-4nnbg"
Jul 07 11:23:37.383725 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:37.383682    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4sz8\" (UniqueName: \"kubernetes.io/projected/b81abf77-b887-4ef6-9b9d-b6c1f94277db-kube-api-access-x4sz8\") pod \"authorino-55655b5696-4nnbg\" (UID: \"b81abf77-b887-4ef6-9b9d-b6c1f94277db\") " pod="kuadrant-system/authorino-55655b5696-4nnbg"
Jul 07 11:23:37.383912 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:37.383754    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b81abf77-b887-4ef6-9b9d-b6c1f94277db-tls-cert\") pod \"authorino-55655b5696-4nnbg\" (UID: \"b81abf77-b887-4ef6-9b9d-b6c1f94277db\") " pod="kuadrant-system/authorino-55655b5696-4nnbg"
Jul 07 11:23:37.383912 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:37.383790    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/b81abf77-b887-4ef6-9b9d-b6c1f94277db-oidc-ca\") pod \"authorino-55655b5696-4nnbg\" (UID: \"b81abf77-b887-4ef6-9b9d-b6c1f94277db\") " pod="kuadrant-system/authorino-55655b5696-4nnbg"
Jul 07 11:23:37.384449 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:37.384432    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/b81abf77-b887-4ef6-9b9d-b6c1f94277db-oidc-ca\") pod \"authorino-55655b5696-4nnbg\" (UID: \"b81abf77-b887-4ef6-9b9d-b6c1f94277db\") " pod="kuadrant-system/authorino-55655b5696-4nnbg"
Jul 07 11:23:37.386380 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:37.386353    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b81abf77-b887-4ef6-9b9d-b6c1f94277db-tls-cert\") pod \"authorino-55655b5696-4nnbg\" (UID: \"b81abf77-b887-4ef6-9b9d-b6c1f94277db\") " pod="kuadrant-system/authorino-55655b5696-4nnbg"
Jul 07 11:23:37.397449 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:37.397426    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4sz8\" (UniqueName: \"kubernetes.io/projected/b81abf77-b887-4ef6-9b9d-b6c1f94277db-kube-api-access-x4sz8\") pod \"authorino-55655b5696-4nnbg\" (UID: \"b81abf77-b887-4ef6-9b9d-b6c1f94277db\") " pod="kuadrant-system/authorino-55655b5696-4nnbg"
Jul 07 11:23:37.502597 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:37.502557    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55655b5696-4nnbg"
Jul 07 11:23:37.655283 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:37.655247    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-55655b5696-4nnbg"]
Jul 07 11:23:37.656252 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:23:37.656223    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb81abf77_b887_4ef6_9b9d_b6c1f94277db.slice/crio-ff057e1dfc3a2783cd74b2ef894775ba7b735b6d66da5a88463718c16f91e878 WatchSource:0}: Error finding container ff057e1dfc3a2783cd74b2ef894775ba7b735b6d66da5a88463718c16f91e878: Status 404 returned error can't find the container with id ff057e1dfc3a2783cd74b2ef894775ba7b735b6d66da5a88463718c16f91e878
Jul 07 11:23:38.329723 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:38.329692    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-55655b5696-4nnbg" event={"ID":"b81abf77-b887-4ef6-9b9d-b6c1f94277db","Type":"ContainerStarted","Data":"ff057e1dfc3a2783cd74b2ef894775ba7b735b6d66da5a88463718c16f91e878"}
Jul 07 11:23:39.339376 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:39.339307    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-55655b5696-4nnbg" event={"ID":"b81abf77-b887-4ef6-9b9d-b6c1f94277db","Type":"ContainerStarted","Data":"b4cdf7cbfda29a63d4b76f2c8c44984eeb7234a3aa744b52c68e8afde8ab1427"}
Jul 07 11:23:39.530274 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:39.530213    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-55655b5696-4nnbg" podStartSLOduration=1.897671201 podStartE2EDuration="2.530194219s" totalImagesPullingTime="632.523018ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:23:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:23:39.511213755 +0000 UTC m=+769.221110143" watchObservedRunningTime="2026-07-07 11:23:39.530194219 +0000 UTC m=+769.240090675"
Jul 07 11:23:39.770906 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:39.770861    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-d6d849cd9-lwkds"]
Jul 07 11:23:39.771155 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:39.771096    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-d6d849cd9-lwkds" podUID="a077d512-dd6f-41a4-857a-ec1b83467ebf" containerName="authorino" containerID="cri-o://e68e28ef4d1580c7dc06e074741320c54f7eca1e5c60ec06011fd821bf4458af" gracePeriod=30
Jul 07 11:23:40.062021 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:40.061986    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d6d849cd9-lwkds"
Jul 07 11:23:40.109066 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:40.109022    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/a077d512-dd6f-41a4-857a-ec1b83467ebf-oidc-ca\") pod \"a077d512-dd6f-41a4-857a-ec1b83467ebf\" (UID: \"a077d512-dd6f-41a4-857a-ec1b83467ebf\") "
Jul 07 11:23:40.109252 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:40.109153    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z6rbq\" (UniqueName: \"kubernetes.io/projected/a077d512-dd6f-41a4-857a-ec1b83467ebf-kube-api-access-z6rbq\") pod \"a077d512-dd6f-41a4-857a-ec1b83467ebf\" (UID: \"a077d512-dd6f-41a4-857a-ec1b83467ebf\") "
Jul 07 11:23:40.109252 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:40.109191    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a077d512-dd6f-41a4-857a-ec1b83467ebf-tls-cert\") pod \"a077d512-dd6f-41a4-857a-ec1b83467ebf\" (UID: \"a077d512-dd6f-41a4-857a-ec1b83467ebf\") "
Jul 07 11:23:40.111977 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:40.111930    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a077d512-dd6f-41a4-857a-ec1b83467ebf-kube-api-access-z6rbq" (OuterVolumeSpecName: "kube-api-access-z6rbq") pod "a077d512-dd6f-41a4-857a-ec1b83467ebf" (UID: "a077d512-dd6f-41a4-857a-ec1b83467ebf"). InnerVolumeSpecName "kube-api-access-z6rbq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:23:40.117615 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:40.117574    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a077d512-dd6f-41a4-857a-ec1b83467ebf-oidc-ca" (OuterVolumeSpecName: "oidc-ca") pod "a077d512-dd6f-41a4-857a-ec1b83467ebf" (UID: "a077d512-dd6f-41a4-857a-ec1b83467ebf"). InnerVolumeSpecName "oidc-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 11:23:40.125406 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:40.125313    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a077d512-dd6f-41a4-857a-ec1b83467ebf-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "a077d512-dd6f-41a4-857a-ec1b83467ebf" (UID: "a077d512-dd6f-41a4-857a-ec1b83467ebf"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 11:23:40.210190 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:40.210152    2570 reconciler_common.go:299] "Volume detached for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/a077d512-dd6f-41a4-857a-ec1b83467ebf-oidc-ca\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:23:40.210190 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:40.210194    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z6rbq\" (UniqueName: \"kubernetes.io/projected/a077d512-dd6f-41a4-857a-ec1b83467ebf-kube-api-access-z6rbq\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:23:40.210401 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:40.210211    2570 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a077d512-dd6f-41a4-857a-ec1b83467ebf-tls-cert\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:23:40.347535 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:40.347431    2570 generic.go:356] "Generic (PLEG): container finished" podID="a077d512-dd6f-41a4-857a-ec1b83467ebf" containerID="e68e28ef4d1580c7dc06e074741320c54f7eca1e5c60ec06011fd821bf4458af" exitCode=0
Jul 07 11:23:40.348170 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:40.348148    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d6d849cd9-lwkds"
Jul 07 11:23:40.348456 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:40.348417    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d6d849cd9-lwkds" event={"ID":"a077d512-dd6f-41a4-857a-ec1b83467ebf","Type":"ContainerDied","Data":"e68e28ef4d1580c7dc06e074741320c54f7eca1e5c60ec06011fd821bf4458af"}
Jul 07 11:23:40.348531 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:40.348499    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d6d849cd9-lwkds" event={"ID":"a077d512-dd6f-41a4-857a-ec1b83467ebf","Type":"ContainerDied","Data":"5aa65d23b12bcf51614a0c81622527499f3cd470216f1fd6a4931a47cb03f93c"}
Jul 07 11:23:40.348531 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:40.348523    2570 scope.go:117] "RemoveContainer" containerID="e68e28ef4d1580c7dc06e074741320c54f7eca1e5c60ec06011fd821bf4458af"
Jul 07 11:23:40.362811 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:40.362788    2570 scope.go:117] "RemoveContainer" containerID="e68e28ef4d1580c7dc06e074741320c54f7eca1e5c60ec06011fd821bf4458af"
Jul 07 11:23:40.363107 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:23:40.363087    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e68e28ef4d1580c7dc06e074741320c54f7eca1e5c60ec06011fd821bf4458af\": container with ID starting with e68e28ef4d1580c7dc06e074741320c54f7eca1e5c60ec06011fd821bf4458af not found: ID does not exist" containerID="e68e28ef4d1580c7dc06e074741320c54f7eca1e5c60ec06011fd821bf4458af"
Jul 07 11:23:40.363182 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:40.363121    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e68e28ef4d1580c7dc06e074741320c54f7eca1e5c60ec06011fd821bf4458af"} err="failed to get container status \"e68e28ef4d1580c7dc06e074741320c54f7eca1e5c60ec06011fd821bf4458af\": rpc error: code = NotFound desc = could not find container \"e68e28ef4d1580c7dc06e074741320c54f7eca1e5c60ec06011fd821bf4458af\": container with ID starting with e68e28ef4d1580c7dc06e074741320c54f7eca1e5c60ec06011fd821bf4458af not found: ID does not exist"
Jul 07 11:23:40.406111 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:40.406073    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-d6d849cd9-lwkds"]
Jul 07 11:23:40.412578 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:40.412542    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-d6d849cd9-lwkds"]
Jul 07 11:23:40.820713 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:23:40.820679    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a077d512-dd6f-41a4-857a-ec1b83467ebf" path="/var/lib/kubelet/pods/a077d512-dd6f-41a4-857a-ec1b83467ebf/volumes"
Jul 07 11:24:50.860625 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:24:50.860590    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7bd7d6c999-qfcp9"]
Jul 07 11:24:50.861091 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:24:50.860787    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-7bd7d6c999-qfcp9" podUID="6c77a500-b555-47d2-a4c4-d5657dac9aad" containerName="manager" containerID="cri-o://0f44ada79754bb6b2078b27cb02e3faafc26d8d1e266687a55c8ad559ff8b5b8" gracePeriod=30
Jul 07 11:24:51.124137 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:24:51.124078    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bd7d6c999-qfcp9"
Jul 07 11:24:51.258726 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:24:51.258694    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6c77a500-b555-47d2-a4c4-d5657dac9aad-webhook-cert\") pod \"6c77a500-b555-47d2-a4c4-d5657dac9aad\" (UID: \"6c77a500-b555-47d2-a4c4-d5657dac9aad\") "
Jul 07 11:24:51.258917 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:24:51.258762    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hzbqg\" (UniqueName: \"kubernetes.io/projected/6c77a500-b555-47d2-a4c4-d5657dac9aad-kube-api-access-hzbqg\") pod \"6c77a500-b555-47d2-a4c4-d5657dac9aad\" (UID: \"6c77a500-b555-47d2-a4c4-d5657dac9aad\") "
Jul 07 11:24:51.260904 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:24:51.260869    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c77a500-b555-47d2-a4c4-d5657dac9aad-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "6c77a500-b555-47d2-a4c4-d5657dac9aad" (UID: "6c77a500-b555-47d2-a4c4-d5657dac9aad"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 11:24:51.261006 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:24:51.260960    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c77a500-b555-47d2-a4c4-d5657dac9aad-kube-api-access-hzbqg" (OuterVolumeSpecName: "kube-api-access-hzbqg") pod "6c77a500-b555-47d2-a4c4-d5657dac9aad" (UID: "6c77a500-b555-47d2-a4c4-d5657dac9aad"). InnerVolumeSpecName "kube-api-access-hzbqg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:24:51.360083 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:24:51.360052    2570 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6c77a500-b555-47d2-a4c4-d5657dac9aad-webhook-cert\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:24:51.360083 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:24:51.360078    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hzbqg\" (UniqueName: \"kubernetes.io/projected/6c77a500-b555-47d2-a4c4-d5657dac9aad-kube-api-access-hzbqg\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:24:51.693303 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:24:51.693268    2570 generic.go:356] "Generic (PLEG): container finished" podID="6c77a500-b555-47d2-a4c4-d5657dac9aad" containerID="0f44ada79754bb6b2078b27cb02e3faafc26d8d1e266687a55c8ad559ff8b5b8" exitCode=0
Jul 07 11:24:51.693478 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:24:51.693315    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bd7d6c999-qfcp9"
Jul 07 11:24:51.693478 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:24:51.693354    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bd7d6c999-qfcp9" event={"ID":"6c77a500-b555-47d2-a4c4-d5657dac9aad","Type":"ContainerDied","Data":"0f44ada79754bb6b2078b27cb02e3faafc26d8d1e266687a55c8ad559ff8b5b8"}
Jul 07 11:24:51.693478 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:24:51.693405    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bd7d6c999-qfcp9" event={"ID":"6c77a500-b555-47d2-a4c4-d5657dac9aad","Type":"ContainerDied","Data":"6d333adc70df17e1aad299b545c16d6b0a0498b9c193d7f963326c70f65a8da1"}
Jul 07 11:24:51.693478 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:24:51.693429    2570 scope.go:117] "RemoveContainer" containerID="0f44ada79754bb6b2078b27cb02e3faafc26d8d1e266687a55c8ad559ff8b5b8"
Jul 07 11:24:51.704945 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:24:51.704929    2570 scope.go:117] "RemoveContainer" containerID="0f44ada79754bb6b2078b27cb02e3faafc26d8d1e266687a55c8ad559ff8b5b8"
Jul 07 11:24:51.705189 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:24:51.705173    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f44ada79754bb6b2078b27cb02e3faafc26d8d1e266687a55c8ad559ff8b5b8\": container with ID starting with 0f44ada79754bb6b2078b27cb02e3faafc26d8d1e266687a55c8ad559ff8b5b8 not found: ID does not exist" containerID="0f44ada79754bb6b2078b27cb02e3faafc26d8d1e266687a55c8ad559ff8b5b8"
Jul 07 11:24:51.705234 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:24:51.705197    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f44ada79754bb6b2078b27cb02e3faafc26d8d1e266687a55c8ad559ff8b5b8"} err="failed to get container status \"0f44ada79754bb6b2078b27cb02e3faafc26d8d1e266687a55c8ad559ff8b5b8\": rpc error: code = NotFound desc = could not find container \"0f44ada79754bb6b2078b27cb02e3faafc26d8d1e266687a55c8ad559ff8b5b8\": container with ID starting with 0f44ada79754bb6b2078b27cb02e3faafc26d8d1e266687a55c8ad559ff8b5b8 not found: ID does not exist"
Jul 07 11:24:51.718060 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:24:51.718035    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7bd7d6c999-qfcp9"]
Jul 07 11:24:51.724457 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:24:51.724438    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7bd7d6c999-qfcp9"]
Jul 07 11:24:52.485270 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:24:52.485238    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7bd7d6c999-df6bp"]
Jul 07 11:24:52.485752 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:24:52.485644    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c77a500-b555-47d2-a4c4-d5657dac9aad" containerName="manager"
Jul 07 11:24:52.485752 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:24:52.485660    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c77a500-b555-47d2-a4c4-d5657dac9aad" containerName="manager"
Jul 07 11:24:52.485752 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:24:52.485686    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a077d512-dd6f-41a4-857a-ec1b83467ebf" containerName="authorino"
Jul 07 11:24:52.485752 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:24:52.485693    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="a077d512-dd6f-41a4-857a-ec1b83467ebf" containerName="authorino"
Jul 07 11:24:52.485915 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:24:52.485775    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c77a500-b555-47d2-a4c4-d5657dac9aad" containerName="manager"
Jul 07 11:24:52.485915 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:24:52.485787    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="a077d512-dd6f-41a4-857a-ec1b83467ebf" containerName="authorino"
Jul 07 11:24:52.489959 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:24:52.489940    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bd7d6c999-df6bp"
Jul 07 11:24:52.495478 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:24:52.495453    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 07 11:24:52.495630 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:24:52.495458    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 07 11:24:52.495630 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:24:52.495461    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-qlnrl\""
Jul 07 11:24:52.500473 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:24:52.500450    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7bd7d6c999-df6bp"]
Jul 07 11:24:52.672574 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:24:52.672539    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plxsx\" (UniqueName: \"kubernetes.io/projected/621d0ca4-4868-4d28-b7db-51e4a459338c-kube-api-access-plxsx\") pod \"maas-controller-7bd7d6c999-df6bp\" (UID: \"621d0ca4-4868-4d28-b7db-51e4a459338c\") " pod="opendatahub/maas-controller-7bd7d6c999-df6bp"
Jul 07 11:24:52.672739 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:24:52.672621    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/621d0ca4-4868-4d28-b7db-51e4a459338c-webhook-cert\") pod \"maas-controller-7bd7d6c999-df6bp\" (UID: \"621d0ca4-4868-4d28-b7db-51e4a459338c\") " pod="opendatahub/maas-controller-7bd7d6c999-df6bp"
Jul 07 11:24:52.773586 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:24:52.773495    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/621d0ca4-4868-4d28-b7db-51e4a459338c-webhook-cert\") pod \"maas-controller-7bd7d6c999-df6bp\" (UID: \"621d0ca4-4868-4d28-b7db-51e4a459338c\") " pod="opendatahub/maas-controller-7bd7d6c999-df6bp"
Jul 07 11:24:52.773586 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:24:52.773579    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plxsx\" (UniqueName: \"kubernetes.io/projected/621d0ca4-4868-4d28-b7db-51e4a459338c-kube-api-access-plxsx\") pod \"maas-controller-7bd7d6c999-df6bp\" (UID: \"621d0ca4-4868-4d28-b7db-51e4a459338c\") " pod="opendatahub/maas-controller-7bd7d6c999-df6bp"
Jul 07 11:24:52.776185 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:24:52.776148    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/621d0ca4-4868-4d28-b7db-51e4a459338c-webhook-cert\") pod \"maas-controller-7bd7d6c999-df6bp\" (UID: \"621d0ca4-4868-4d28-b7db-51e4a459338c\") " pod="opendatahub/maas-controller-7bd7d6c999-df6bp"
Jul 07 11:24:52.785514 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:24:52.785487    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-plxsx\" (UniqueName: \"kubernetes.io/projected/621d0ca4-4868-4d28-b7db-51e4a459338c-kube-api-access-plxsx\") pod \"maas-controller-7bd7d6c999-df6bp\" (UID: \"621d0ca4-4868-4d28-b7db-51e4a459338c\") " pod="opendatahub/maas-controller-7bd7d6c999-df6bp"
Jul 07 11:24:52.800118 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:24:52.800091    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bd7d6c999-df6bp"
Jul 07 11:24:52.821077 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:24:52.821045    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c77a500-b555-47d2-a4c4-d5657dac9aad" path="/var/lib/kubelet/pods/6c77a500-b555-47d2-a4c4-d5657dac9aad/volumes"
Jul 07 11:24:52.951948 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:24:52.951910    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7bd7d6c999-df6bp"]
Jul 07 11:24:52.953366 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:24:52.953316    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod621d0ca4_4868_4d28_b7db_51e4a459338c.slice/crio-89f20281d2e1ce6fb12bd3831b5cc40574c9983ca42353847ff61552b7fdacdf WatchSource:0}: Error finding container 89f20281d2e1ce6fb12bd3831b5cc40574c9983ca42353847ff61552b7fdacdf: Status 404 returned error can't find the container with id 89f20281d2e1ce6fb12bd3831b5cc40574c9983ca42353847ff61552b7fdacdf
Jul 07 11:24:53.711045 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:24:53.710938    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bd7d6c999-df6bp" event={"ID":"621d0ca4-4868-4d28-b7db-51e4a459338c","Type":"ContainerStarted","Data":"d816a6e4191c72461208f919bdacf924625082d2db245fc5650a6ffce96b31cf"}
Jul 07 11:24:53.711045 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:24:53.711010    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bd7d6c999-df6bp" event={"ID":"621d0ca4-4868-4d28-b7db-51e4a459338c","Type":"ContainerStarted","Data":"89f20281d2e1ce6fb12bd3831b5cc40574c9983ca42353847ff61552b7fdacdf"}
Jul 07 11:24:53.711516 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:24:53.711064    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7bd7d6c999-df6bp"
Jul 07 11:24:53.736027 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:24:53.735979    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7bd7d6c999-df6bp" podStartSLOduration=1.2757650649999999 podStartE2EDuration="1.735964824s" totalImagesPullingTime="460.199759ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:24:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:24:53.735504818 +0000 UTC m=+843.445401207" watchObservedRunningTime="2026-07-07 11:24:53.735964824 +0000 UTC m=+843.445861245"
Jul 07 11:25:04.722504 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:25:04.722463    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7bd7d6c999-df6bp"
Jul 07 11:25:50.773719 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:25:50.773636    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4v7v8_c8c17c90-3e07-4530-8411-d59cef9a1aff/console-operator/2.log"
Jul 07 11:25:50.775075 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:25:50.775050    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4v7v8_c8c17c90-3e07-4530-8411-d59cef9a1aff/console-operator/2.log"
Jul 07 11:30:50.807644 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:30:50.807609    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4v7v8_c8c17c90-3e07-4530-8411-d59cef9a1aff/console-operator/2.log"
Jul 07 11:30:50.808183 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:30:50.808165    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4v7v8_c8c17c90-3e07-4530-8411-d59cef9a1aff/console-operator/2.log"
Jul 07 11:35:50.836440 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:35:50.836414    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4v7v8_c8c17c90-3e07-4530-8411-d59cef9a1aff/console-operator/2.log"
Jul 07 11:35:50.837301 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:35:50.837279    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4v7v8_c8c17c90-3e07-4530-8411-d59cef9a1aff/console-operator/2.log"
Jul 07 11:36:38.968306 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:38.968262    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-90c5983b-b78b84c5b-jnxvl"]
Jul 07 11:36:38.971908 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:38.971891    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-90c5983b-b78b84c5b-jnxvl"
Jul 07 11:36:38.976942 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:38.976916    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-90c5983b-serving-cert\""
Jul 07 11:36:38.994967 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:38.994936    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-90c5983b-b78b84c5b-jnxvl"]
Jul 07 11:36:39.076043 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:39.076004    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69793f40-b556-495f-8609-77e1b17cd8aa-openshift-service-ca\") pod \"maas-api-e2e-ait-90c5983b-b78b84c5b-jnxvl\" (UID: \"69793f40-b556-495f-8609-77e1b17cd8aa\") " pod="opendatahub/maas-api-e2e-ait-90c5983b-b78b84c5b-jnxvl"
Jul 07 11:36:39.076043 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:39.076046    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69793f40-b556-495f-8609-77e1b17cd8aa-maas-api-tls\") pod \"maas-api-e2e-ait-90c5983b-b78b84c5b-jnxvl\" (UID: \"69793f40-b556-495f-8609-77e1b17cd8aa\") " pod="opendatahub/maas-api-e2e-ait-90c5983b-b78b84c5b-jnxvl"
Jul 07 11:36:39.076299 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:39.076143    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hq7zh\" (UniqueName: \"kubernetes.io/projected/69793f40-b556-495f-8609-77e1b17cd8aa-kube-api-access-hq7zh\") pod \"maas-api-e2e-ait-90c5983b-b78b84c5b-jnxvl\" (UID: \"69793f40-b556-495f-8609-77e1b17cd8aa\") " pod="opendatahub/maas-api-e2e-ait-90c5983b-b78b84c5b-jnxvl"
Jul 07 11:36:39.177590 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:39.177555    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hq7zh\" (UniqueName: \"kubernetes.io/projected/69793f40-b556-495f-8609-77e1b17cd8aa-kube-api-access-hq7zh\") pod \"maas-api-e2e-ait-90c5983b-b78b84c5b-jnxvl\" (UID: \"69793f40-b556-495f-8609-77e1b17cd8aa\") " pod="opendatahub/maas-api-e2e-ait-90c5983b-b78b84c5b-jnxvl"
Jul 07 11:36:39.177773 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:39.177618    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69793f40-b556-495f-8609-77e1b17cd8aa-openshift-service-ca\") pod \"maas-api-e2e-ait-90c5983b-b78b84c5b-jnxvl\" (UID: \"69793f40-b556-495f-8609-77e1b17cd8aa\") " pod="opendatahub/maas-api-e2e-ait-90c5983b-b78b84c5b-jnxvl"
Jul 07 11:36:39.177773 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:39.177644    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69793f40-b556-495f-8609-77e1b17cd8aa-maas-api-tls\") pod \"maas-api-e2e-ait-90c5983b-b78b84c5b-jnxvl\" (UID: \"69793f40-b556-495f-8609-77e1b17cd8aa\") " pod="opendatahub/maas-api-e2e-ait-90c5983b-b78b84c5b-jnxvl"
Jul 07 11:36:39.178453 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:39.178306    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69793f40-b556-495f-8609-77e1b17cd8aa-openshift-service-ca\") pod \"maas-api-e2e-ait-90c5983b-b78b84c5b-jnxvl\" (UID: \"69793f40-b556-495f-8609-77e1b17cd8aa\") " pod="opendatahub/maas-api-e2e-ait-90c5983b-b78b84c5b-jnxvl"
Jul 07 11:36:39.180100 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:39.180078    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69793f40-b556-495f-8609-77e1b17cd8aa-maas-api-tls\") pod \"maas-api-e2e-ait-90c5983b-b78b84c5b-jnxvl\" (UID: \"69793f40-b556-495f-8609-77e1b17cd8aa\") " pod="opendatahub/maas-api-e2e-ait-90c5983b-b78b84c5b-jnxvl"
Jul 07 11:36:39.209150 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:39.209107    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hq7zh\" (UniqueName: \"kubernetes.io/projected/69793f40-b556-495f-8609-77e1b17cd8aa-kube-api-access-hq7zh\") pod \"maas-api-e2e-ait-90c5983b-b78b84c5b-jnxvl\" (UID: \"69793f40-b556-495f-8609-77e1b17cd8aa\") " pod="opendatahub/maas-api-e2e-ait-90c5983b-b78b84c5b-jnxvl"
Jul 07 11:36:39.281840 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:39.281807    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-90c5983b-b78b84c5b-jnxvl"
Jul 07 11:36:39.447166 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:36:39.447140    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69793f40_b556_495f_8609_77e1b17cd8aa.slice/crio-6837fcf40f805818b37376eb74d238b5cc08f20af76996da047476bd95f81607 WatchSource:0}: Error finding container 6837fcf40f805818b37376eb74d238b5cc08f20af76996da047476bd95f81607: Status 404 returned error can't find the container with id 6837fcf40f805818b37376eb74d238b5cc08f20af76996da047476bd95f81607
Jul 07 11:36:39.447541 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:39.447517    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-90c5983b-b78b84c5b-jnxvl"]
Jul 07 11:36:39.448760 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:39.448737    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 11:36:40.130298 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:40.130238    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-90c5983b-b78b84c5b-jnxvl" event={"ID":"69793f40-b556-495f-8609-77e1b17cd8aa","Type":"ContainerStarted","Data":"c051f4aa248918c3a63cb25f96187ad3c9f5c87fcf7569ebe25c41d23b21bbaf"}
Jul 07 11:36:40.130750 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:40.130313    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-90c5983b-b78b84c5b-jnxvl" event={"ID":"69793f40-b556-495f-8609-77e1b17cd8aa","Type":"ContainerStarted","Data":"6837fcf40f805818b37376eb74d238b5cc08f20af76996da047476bd95f81607"}
Jul 07 11:36:40.130750 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:40.130358    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-90c5983b-b78b84c5b-jnxvl"
Jul 07 11:36:40.156718 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:40.156661    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-90c5983b-b78b84c5b-jnxvl" podStartSLOduration=1.6557679840000001 podStartE2EDuration="2.156639625s" totalImagesPullingTime="500.871641ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:36:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:36:40.152578021 +0000 UTC m=+1549.862474408" watchObservedRunningTime="2026-07-07 11:36:40.156639625 +0000 UTC m=+1549.866536012"
Jul 07 11:36:44.131964 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:44.131933    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-90c5983b-b78b84c5b-jnxvl"]
Jul 07 11:36:44.132520 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:44.132101    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-90c5983b-b78b84c5b-jnxvl" podUID="69793f40-b556-495f-8609-77e1b17cd8aa" containerName="maas-api" containerID="cri-o://c051f4aa248918c3a63cb25f96187ad3c9f5c87fcf7569ebe25c41d23b21bbaf" gracePeriod=30
Jul 07 11:36:44.386643 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:44.386581    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-90c5983b-b78b84c5b-jnxvl"
Jul 07 11:36:44.424940 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:44.424890    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69793f40-b556-495f-8609-77e1b17cd8aa-openshift-service-ca\") pod \"69793f40-b556-495f-8609-77e1b17cd8aa\" (UID: \"69793f40-b556-495f-8609-77e1b17cd8aa\") "
Jul 07 11:36:44.425122 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:44.424996    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69793f40-b556-495f-8609-77e1b17cd8aa-maas-api-tls\") pod \"69793f40-b556-495f-8609-77e1b17cd8aa\" (UID: \"69793f40-b556-495f-8609-77e1b17cd8aa\") "
Jul 07 11:36:44.425122 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:44.425058    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hq7zh\" (UniqueName: \"kubernetes.io/projected/69793f40-b556-495f-8609-77e1b17cd8aa-kube-api-access-hq7zh\") pod \"69793f40-b556-495f-8609-77e1b17cd8aa\" (UID: \"69793f40-b556-495f-8609-77e1b17cd8aa\") "
Jul 07 11:36:44.425244 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:44.425221    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69793f40-b556-495f-8609-77e1b17cd8aa-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "69793f40-b556-495f-8609-77e1b17cd8aa" (UID: "69793f40-b556-495f-8609-77e1b17cd8aa"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 11:36:44.425343 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:44.425314    2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69793f40-b556-495f-8609-77e1b17cd8aa-openshift-service-ca\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:36:44.427251 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:44.427219    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69793f40-b556-495f-8609-77e1b17cd8aa-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "69793f40-b556-495f-8609-77e1b17cd8aa" (UID: "69793f40-b556-495f-8609-77e1b17cd8aa"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 11:36:44.427390 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:44.427370    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69793f40-b556-495f-8609-77e1b17cd8aa-kube-api-access-hq7zh" (OuterVolumeSpecName: "kube-api-access-hq7zh") pod "69793f40-b556-495f-8609-77e1b17cd8aa" (UID: "69793f40-b556-495f-8609-77e1b17cd8aa"). InnerVolumeSpecName "kube-api-access-hq7zh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:36:44.525909 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:44.525869    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69793f40-b556-495f-8609-77e1b17cd8aa-maas-api-tls\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:36:44.525909 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:44.525905    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hq7zh\" (UniqueName: \"kubernetes.io/projected/69793f40-b556-495f-8609-77e1b17cd8aa-kube-api-access-hq7zh\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:36:45.154392 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:45.154290    2570 generic.go:356] "Generic (PLEG): container finished" podID="69793f40-b556-495f-8609-77e1b17cd8aa" containerID="c051f4aa248918c3a63cb25f96187ad3c9f5c87fcf7569ebe25c41d23b21bbaf" exitCode=0
Jul 07 11:36:45.154392 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:45.154355    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-90c5983b-b78b84c5b-jnxvl"
Jul 07 11:36:45.154852 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:45.154354    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-90c5983b-b78b84c5b-jnxvl" event={"ID":"69793f40-b556-495f-8609-77e1b17cd8aa","Type":"ContainerDied","Data":"c051f4aa248918c3a63cb25f96187ad3c9f5c87fcf7569ebe25c41d23b21bbaf"}
Jul 07 11:36:45.154852 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:45.154491    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-90c5983b-b78b84c5b-jnxvl" event={"ID":"69793f40-b556-495f-8609-77e1b17cd8aa","Type":"ContainerDied","Data":"6837fcf40f805818b37376eb74d238b5cc08f20af76996da047476bd95f81607"}
Jul 07 11:36:45.154852 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:45.154522    2570 scope.go:117] "RemoveContainer" containerID="c051f4aa248918c3a63cb25f96187ad3c9f5c87fcf7569ebe25c41d23b21bbaf"
Jul 07 11:36:45.163378 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:45.163315    2570 scope.go:117] "RemoveContainer" containerID="c051f4aa248918c3a63cb25f96187ad3c9f5c87fcf7569ebe25c41d23b21bbaf"
Jul 07 11:36:45.163617 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:36:45.163596    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c051f4aa248918c3a63cb25f96187ad3c9f5c87fcf7569ebe25c41d23b21bbaf\": container with ID starting with c051f4aa248918c3a63cb25f96187ad3c9f5c87fcf7569ebe25c41d23b21bbaf not found: ID does not exist" containerID="c051f4aa248918c3a63cb25f96187ad3c9f5c87fcf7569ebe25c41d23b21bbaf"
Jul 07 11:36:45.163687 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:45.163631    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c051f4aa248918c3a63cb25f96187ad3c9f5c87fcf7569ebe25c41d23b21bbaf"} err="failed to get container status \"c051f4aa248918c3a63cb25f96187ad3c9f5c87fcf7569ebe25c41d23b21bbaf\": rpc error: code = NotFound desc = could not find container \"c051f4aa248918c3a63cb25f96187ad3c9f5c87fcf7569ebe25c41d23b21bbaf\": container with ID starting with c051f4aa248918c3a63cb25f96187ad3c9f5c87fcf7569ebe25c41d23b21bbaf not found: ID does not exist"
Jul 07 11:36:45.173127 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:45.173107    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-90c5983b-b78b84c5b-jnxvl"]
Jul 07 11:36:45.175714 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:45.175694    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-90c5983b-b78b84c5b-jnxvl"]
Jul 07 11:36:46.820619 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:46.820578    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69793f40-b556-495f-8609-77e1b17cd8aa" path="/var/lib/kubelet/pods/69793f40-b556-495f-8609-77e1b17cd8aa/volumes"
Jul 07 11:36:51.934136 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:51.934090    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-7b29c9bf-5d79bdcff9-2w8wg"]
Jul 07 11:36:51.934703 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:51.934682    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69793f40-b556-495f-8609-77e1b17cd8aa" containerName="maas-api"
Jul 07 11:36:51.934789 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:51.934705    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="69793f40-b556-495f-8609-77e1b17cd8aa" containerName="maas-api"
Jul 07 11:36:51.934841 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:51.934811    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="69793f40-b556-495f-8609-77e1b17cd8aa" containerName="maas-api"
Jul 07 11:36:51.938288 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:51.938266    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-7b29c9bf-5d79bdcff9-2w8wg"
Jul 07 11:36:51.941142 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:51.941118    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-7b29c9bf-serving-cert\""
Jul 07 11:36:51.947030 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:51.947008    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-7b29c9bf-5d79bdcff9-2w8wg"]
Jul 07 11:36:51.994366 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:51.994300    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f419dec6-0885-44f0-8603-42953c3a24ec-openshift-service-ca\") pod \"maas-api-e2e-ait-7b29c9bf-5d79bdcff9-2w8wg\" (UID: \"f419dec6-0885-44f0-8603-42953c3a24ec\") " pod="opendatahub/maas-api-e2e-ait-7b29c9bf-5d79bdcff9-2w8wg"
Jul 07 11:36:51.994558 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:51.994426    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2gtz\" (UniqueName: \"kubernetes.io/projected/f419dec6-0885-44f0-8603-42953c3a24ec-kube-api-access-d2gtz\") pod \"maas-api-e2e-ait-7b29c9bf-5d79bdcff9-2w8wg\" (UID: \"f419dec6-0885-44f0-8603-42953c3a24ec\") " pod="opendatahub/maas-api-e2e-ait-7b29c9bf-5d79bdcff9-2w8wg"
Jul 07 11:36:51.994558 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:51.994491    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f419dec6-0885-44f0-8603-42953c3a24ec-maas-api-tls\") pod \"maas-api-e2e-ait-7b29c9bf-5d79bdcff9-2w8wg\" (UID: \"f419dec6-0885-44f0-8603-42953c3a24ec\") " pod="opendatahub/maas-api-e2e-ait-7b29c9bf-5d79bdcff9-2w8wg"
Jul 07 11:36:52.095740 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:52.095708    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f419dec6-0885-44f0-8603-42953c3a24ec-openshift-service-ca\") pod \"maas-api-e2e-ait-7b29c9bf-5d79bdcff9-2w8wg\" (UID: \"f419dec6-0885-44f0-8603-42953c3a24ec\") " pod="opendatahub/maas-api-e2e-ait-7b29c9bf-5d79bdcff9-2w8wg"
Jul 07 11:36:52.095928 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:52.095754    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2gtz\" (UniqueName: \"kubernetes.io/projected/f419dec6-0885-44f0-8603-42953c3a24ec-kube-api-access-d2gtz\") pod \"maas-api-e2e-ait-7b29c9bf-5d79bdcff9-2w8wg\" (UID: \"f419dec6-0885-44f0-8603-42953c3a24ec\") " pod="opendatahub/maas-api-e2e-ait-7b29c9bf-5d79bdcff9-2w8wg"
Jul 07 11:36:52.095928 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:52.095789    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f419dec6-0885-44f0-8603-42953c3a24ec-maas-api-tls\") pod \"maas-api-e2e-ait-7b29c9bf-5d79bdcff9-2w8wg\" (UID: \"f419dec6-0885-44f0-8603-42953c3a24ec\") " pod="opendatahub/maas-api-e2e-ait-7b29c9bf-5d79bdcff9-2w8wg"
Jul 07 11:36:52.095928 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:36:52.095901    2570 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-7b29c9bf-serving-cert: secret "maas-api-e2e-ait-7b29c9bf-serving-cert" not found
Jul 07 11:36:52.096105 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:36:52.095955    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f419dec6-0885-44f0-8603-42953c3a24ec-maas-api-tls podName:f419dec6-0885-44f0-8603-42953c3a24ec nodeName:}" failed. No retries permitted until 2026-07-07 11:36:52.595939541 +0000 UTC m=+1562.305835906 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f419dec6-0885-44f0-8603-42953c3a24ec-maas-api-tls") pod "maas-api-e2e-ait-7b29c9bf-5d79bdcff9-2w8wg" (UID: "f419dec6-0885-44f0-8603-42953c3a24ec") : secret "maas-api-e2e-ait-7b29c9bf-serving-cert" not found
Jul 07 11:36:52.096452 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:52.096426    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f419dec6-0885-44f0-8603-42953c3a24ec-openshift-service-ca\") pod \"maas-api-e2e-ait-7b29c9bf-5d79bdcff9-2w8wg\" (UID: \"f419dec6-0885-44f0-8603-42953c3a24ec\") " pod="opendatahub/maas-api-e2e-ait-7b29c9bf-5d79bdcff9-2w8wg"
Jul 07 11:36:52.112710 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:52.112678    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2gtz\" (UniqueName: \"kubernetes.io/projected/f419dec6-0885-44f0-8603-42953c3a24ec-kube-api-access-d2gtz\") pod \"maas-api-e2e-ait-7b29c9bf-5d79bdcff9-2w8wg\" (UID: \"f419dec6-0885-44f0-8603-42953c3a24ec\") " pod="opendatahub/maas-api-e2e-ait-7b29c9bf-5d79bdcff9-2w8wg"
Jul 07 11:36:52.602348 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:52.602296    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f419dec6-0885-44f0-8603-42953c3a24ec-maas-api-tls\") pod \"maas-api-e2e-ait-7b29c9bf-5d79bdcff9-2w8wg\" (UID: \"f419dec6-0885-44f0-8603-42953c3a24ec\") " pod="opendatahub/maas-api-e2e-ait-7b29c9bf-5d79bdcff9-2w8wg"
Jul 07 11:36:52.609307 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:52.609277    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f419dec6-0885-44f0-8603-42953c3a24ec-maas-api-tls\") pod \"maas-api-e2e-ait-7b29c9bf-5d79bdcff9-2w8wg\" (UID: \"f419dec6-0885-44f0-8603-42953c3a24ec\") " pod="opendatahub/maas-api-e2e-ait-7b29c9bf-5d79bdcff9-2w8wg"
Jul 07 11:36:52.850396 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:52.850348    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-7b29c9bf-5d79bdcff9-2w8wg"
Jul 07 11:36:52.995972 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:52.995941    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-7b29c9bf-5d79bdcff9-2w8wg"]
Jul 07 11:36:52.997872 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:36:52.997846    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf419dec6_0885_44f0_8603_42953c3a24ec.slice/crio-c0c313b52a098fad5aa8c37649c82e037575276482c31533606af547385f83c6 WatchSource:0}: Error finding container c0c313b52a098fad5aa8c37649c82e037575276482c31533606af547385f83c6: Status 404 returned error can't find the container with id c0c313b52a098fad5aa8c37649c82e037575276482c31533606af547385f83c6
Jul 07 11:36:53.194031 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:53.193941    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-7b29c9bf-5d79bdcff9-2w8wg" event={"ID":"f419dec6-0885-44f0-8603-42953c3a24ec","Type":"ContainerStarted","Data":"c0c313b52a098fad5aa8c37649c82e037575276482c31533606af547385f83c6"}
Jul 07 11:36:54.200817 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:54.200771    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-7b29c9bf-5d79bdcff9-2w8wg" event={"ID":"f419dec6-0885-44f0-8603-42953c3a24ec","Type":"ContainerStarted","Data":"563aa5b7724bb72a026ae98a02f002b1028e45357abedaff44168ec225953106"}
Jul 07 11:36:54.201227 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:54.200910    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-7b29c9bf-5d79bdcff9-2w8wg"
Jul 07 11:36:54.231567 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:54.231517    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-7b29c9bf-5d79bdcff9-2w8wg" podStartSLOduration=2.8661540309999998 podStartE2EDuration="3.231499444s" totalImagesPullingTime="365.345413ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:36:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:36:54.22897771 +0000 UTC m=+1563.938874098" watchObservedRunningTime="2026-07-07 11:36:54.231499444 +0000 UTC m=+1563.941395831"
Jul 07 11:36:57.349859 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:57.349822    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-7b29c9bf-5d79bdcff9-2w8wg"]
Jul 07 11:36:57.350317 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:57.350061    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-7b29c9bf-5d79bdcff9-2w8wg" podUID="f419dec6-0885-44f0-8603-42953c3a24ec" containerName="maas-api" containerID="cri-o://563aa5b7724bb72a026ae98a02f002b1028e45357abedaff44168ec225953106" gracePeriod=30
Jul 07 11:36:57.622049 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:57.622019    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-7b29c9bf-5d79bdcff9-2w8wg"
Jul 07 11:36:57.752167 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:57.752119    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d2gtz\" (UniqueName: \"kubernetes.io/projected/f419dec6-0885-44f0-8603-42953c3a24ec-kube-api-access-d2gtz\") pod \"f419dec6-0885-44f0-8603-42953c3a24ec\" (UID: \"f419dec6-0885-44f0-8603-42953c3a24ec\") "
Jul 07 11:36:57.752380 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:57.752229    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f419dec6-0885-44f0-8603-42953c3a24ec-maas-api-tls\") pod \"f419dec6-0885-44f0-8603-42953c3a24ec\" (UID: \"f419dec6-0885-44f0-8603-42953c3a24ec\") "
Jul 07 11:36:57.752380 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:57.752260    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f419dec6-0885-44f0-8603-42953c3a24ec-openshift-service-ca\") pod \"f419dec6-0885-44f0-8603-42953c3a24ec\" (UID: \"f419dec6-0885-44f0-8603-42953c3a24ec\") "
Jul 07 11:36:57.752664 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:57.752633    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f419dec6-0885-44f0-8603-42953c3a24ec-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f419dec6-0885-44f0-8603-42953c3a24ec" (UID: "f419dec6-0885-44f0-8603-42953c3a24ec"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 11:36:57.754559 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:57.754526    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f419dec6-0885-44f0-8603-42953c3a24ec-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f419dec6-0885-44f0-8603-42953c3a24ec" (UID: "f419dec6-0885-44f0-8603-42953c3a24ec"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 11:36:57.754559 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:57.754527    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f419dec6-0885-44f0-8603-42953c3a24ec-kube-api-access-d2gtz" (OuterVolumeSpecName: "kube-api-access-d2gtz") pod "f419dec6-0885-44f0-8603-42953c3a24ec" (UID: "f419dec6-0885-44f0-8603-42953c3a24ec"). InnerVolumeSpecName "kube-api-access-d2gtz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:36:57.853227 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:57.853145    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f419dec6-0885-44f0-8603-42953c3a24ec-maas-api-tls\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:36:57.853227 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:57.853173    2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f419dec6-0885-44f0-8603-42953c3a24ec-openshift-service-ca\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:36:57.853227 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:57.853184    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d2gtz\" (UniqueName: \"kubernetes.io/projected/f419dec6-0885-44f0-8603-42953c3a24ec-kube-api-access-d2gtz\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:36:58.221267 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:58.221186    2570 generic.go:356] "Generic (PLEG): container finished" podID="f419dec6-0885-44f0-8603-42953c3a24ec" containerID="563aa5b7724bb72a026ae98a02f002b1028e45357abedaff44168ec225953106" exitCode=0
Jul 07 11:36:58.221267 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:58.221235    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-7b29c9bf-5d79bdcff9-2w8wg"
Jul 07 11:36:58.221467 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:58.221279    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-7b29c9bf-5d79bdcff9-2w8wg" event={"ID":"f419dec6-0885-44f0-8603-42953c3a24ec","Type":"ContainerDied","Data":"563aa5b7724bb72a026ae98a02f002b1028e45357abedaff44168ec225953106"}
Jul 07 11:36:58.221467 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:58.221349    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-7b29c9bf-5d79bdcff9-2w8wg" event={"ID":"f419dec6-0885-44f0-8603-42953c3a24ec","Type":"ContainerDied","Data":"c0c313b52a098fad5aa8c37649c82e037575276482c31533606af547385f83c6"}
Jul 07 11:36:58.221467 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:58.221378    2570 scope.go:117] "RemoveContainer" containerID="563aa5b7724bb72a026ae98a02f002b1028e45357abedaff44168ec225953106"
Jul 07 11:36:58.230687 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:58.230658    2570 scope.go:117] "RemoveContainer" containerID="563aa5b7724bb72a026ae98a02f002b1028e45357abedaff44168ec225953106"
Jul 07 11:36:58.230946 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:36:58.230926    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"563aa5b7724bb72a026ae98a02f002b1028e45357abedaff44168ec225953106\": container with ID starting with 563aa5b7724bb72a026ae98a02f002b1028e45357abedaff44168ec225953106 not found: ID does not exist" containerID="563aa5b7724bb72a026ae98a02f002b1028e45357abedaff44168ec225953106"
Jul 07 11:36:58.231019 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:58.230956    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"563aa5b7724bb72a026ae98a02f002b1028e45357abedaff44168ec225953106"} err="failed to get container status \"563aa5b7724bb72a026ae98a02f002b1028e45357abedaff44168ec225953106\": rpc error: code = NotFound desc = could not find container \"563aa5b7724bb72a026ae98a02f002b1028e45357abedaff44168ec225953106\": container with ID starting with 563aa5b7724bb72a026ae98a02f002b1028e45357abedaff44168ec225953106 not found: ID does not exist"
Jul 07 11:36:58.254841 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:58.254812    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-7b29c9bf-5d79bdcff9-2w8wg"]
Jul 07 11:36:58.261616 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:58.261592    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-7b29c9bf-5d79bdcff9-2w8wg"]
Jul 07 11:36:58.821012 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:36:58.820982    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f419dec6-0885-44f0-8603-42953c3a24ec" path="/var/lib/kubelet/pods/f419dec6-0885-44f0-8603-42953c3a24ec/volumes"
Jul 07 11:37:11.472180 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:37:11.472133    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-derive-8bb53107-6557b66dbd-6224h"]
Jul 07 11:37:11.472741 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:37:11.472720    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f419dec6-0885-44f0-8603-42953c3a24ec" containerName="maas-api"
Jul 07 11:37:11.472821 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:37:11.472744    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="f419dec6-0885-44f0-8603-42953c3a24ec" containerName="maas-api"
Jul 07 11:37:11.472872 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:37:11.472836    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="f419dec6-0885-44f0-8603-42953c3a24ec" containerName="maas-api"
Jul 07 11:37:11.477157 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:37:11.477134    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-8bb53107-6557b66dbd-6224h"
Jul 07 11:37:11.480275 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:37:11.480255    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-derive-8bb53107-serving-cert\""
Jul 07 11:37:11.496943 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:37:11.496901    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-8bb53107-6557b66dbd-6224h"]
Jul 07 11:37:11.574256 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:37:11.574220    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/570ab3a3-e08f-47b8-8273-49852b16ce07-openshift-service-ca\") pod \"maas-api-e2e-derive-8bb53107-6557b66dbd-6224h\" (UID: \"570ab3a3-e08f-47b8-8273-49852b16ce07\") " pod="opendatahub/maas-api-e2e-derive-8bb53107-6557b66dbd-6224h"
Jul 07 11:37:11.574454 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:37:11.574290    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxx6v\" (UniqueName: \"kubernetes.io/projected/570ab3a3-e08f-47b8-8273-49852b16ce07-kube-api-access-qxx6v\") pod \"maas-api-e2e-derive-8bb53107-6557b66dbd-6224h\" (UID: \"570ab3a3-e08f-47b8-8273-49852b16ce07\") " pod="opendatahub/maas-api-e2e-derive-8bb53107-6557b66dbd-6224h"
Jul 07 11:37:11.574454 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:37:11.574417    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/570ab3a3-e08f-47b8-8273-49852b16ce07-maas-api-tls\") pod \"maas-api-e2e-derive-8bb53107-6557b66dbd-6224h\" (UID: \"570ab3a3-e08f-47b8-8273-49852b16ce07\") " pod="opendatahub/maas-api-e2e-derive-8bb53107-6557b66dbd-6224h"
Jul 07 11:37:11.675238 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:37:11.675183    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/570ab3a3-e08f-47b8-8273-49852b16ce07-maas-api-tls\") pod \"maas-api-e2e-derive-8bb53107-6557b66dbd-6224h\" (UID: \"570ab3a3-e08f-47b8-8273-49852b16ce07\") " pod="opendatahub/maas-api-e2e-derive-8bb53107-6557b66dbd-6224h"
Jul 07 11:37:11.675477 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:37:11.675254    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/570ab3a3-e08f-47b8-8273-49852b16ce07-openshift-service-ca\") pod \"maas-api-e2e-derive-8bb53107-6557b66dbd-6224h\" (UID: \"570ab3a3-e08f-47b8-8273-49852b16ce07\") " pod="opendatahub/maas-api-e2e-derive-8bb53107-6557b66dbd-6224h"
Jul 07 11:37:11.675477 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:37:11.675314    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxx6v\" (UniqueName: \"kubernetes.io/projected/570ab3a3-e08f-47b8-8273-49852b16ce07-kube-api-access-qxx6v\") pod \"maas-api-e2e-derive-8bb53107-6557b66dbd-6224h\" (UID: \"570ab3a3-e08f-47b8-8273-49852b16ce07\") " pod="opendatahub/maas-api-e2e-derive-8bb53107-6557b66dbd-6224h"
Jul 07 11:37:11.675477 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:37:11.675358    2570 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-derive-8bb53107-serving-cert: secret "maas-api-e2e-derive-8bb53107-serving-cert" not found
Jul 07 11:37:11.675477 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:37:11.675434    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/570ab3a3-e08f-47b8-8273-49852b16ce07-maas-api-tls podName:570ab3a3-e08f-47b8-8273-49852b16ce07 nodeName:}" failed. No retries permitted until 2026-07-07 11:37:12.175416922 +0000 UTC m=+1581.885313288 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/570ab3a3-e08f-47b8-8273-49852b16ce07-maas-api-tls") pod "maas-api-e2e-derive-8bb53107-6557b66dbd-6224h" (UID: "570ab3a3-e08f-47b8-8273-49852b16ce07") : secret "maas-api-e2e-derive-8bb53107-serving-cert" not found
Jul 07 11:37:11.675963 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:37:11.675940    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/570ab3a3-e08f-47b8-8273-49852b16ce07-openshift-service-ca\") pod \"maas-api-e2e-derive-8bb53107-6557b66dbd-6224h\" (UID: \"570ab3a3-e08f-47b8-8273-49852b16ce07\") " pod="opendatahub/maas-api-e2e-derive-8bb53107-6557b66dbd-6224h"
Jul 07 11:37:11.695761 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:37:11.695719    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxx6v\" (UniqueName: \"kubernetes.io/projected/570ab3a3-e08f-47b8-8273-49852b16ce07-kube-api-access-qxx6v\") pod \"maas-api-e2e-derive-8bb53107-6557b66dbd-6224h\" (UID: \"570ab3a3-e08f-47b8-8273-49852b16ce07\") " pod="opendatahub/maas-api-e2e-derive-8bb53107-6557b66dbd-6224h"
Jul 07 11:37:12.179337 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:37:12.179302    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/570ab3a3-e08f-47b8-8273-49852b16ce07-maas-api-tls\") pod \"maas-api-e2e-derive-8bb53107-6557b66dbd-6224h\" (UID: \"570ab3a3-e08f-47b8-8273-49852b16ce07\") " pod="opendatahub/maas-api-e2e-derive-8bb53107-6557b66dbd-6224h"
Jul 07 11:37:12.181874 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:37:12.181848    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/570ab3a3-e08f-47b8-8273-49852b16ce07-maas-api-tls\") pod \"maas-api-e2e-derive-8bb53107-6557b66dbd-6224h\" (UID: \"570ab3a3-e08f-47b8-8273-49852b16ce07\") " pod="opendatahub/maas-api-e2e-derive-8bb53107-6557b66dbd-6224h"
Jul 07 11:37:12.387116 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:37:12.387067    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-8bb53107-6557b66dbd-6224h"
Jul 07 11:37:12.537201 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:37:12.537106    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-8bb53107-6557b66dbd-6224h"]
Jul 07 11:37:12.539966 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:37:12.539932    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod570ab3a3_e08f_47b8_8273_49852b16ce07.slice/crio-271aafdf15a5cfa95917ebdad3ccfa921a4bb0bba8a1922ba203c0db692e024e WatchSource:0}: Error finding container 271aafdf15a5cfa95917ebdad3ccfa921a4bb0bba8a1922ba203c0db692e024e: Status 404 returned error can't find the container with id 271aafdf15a5cfa95917ebdad3ccfa921a4bb0bba8a1922ba203c0db692e024e
Jul 07 11:37:13.293580 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:37:13.293544    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-8bb53107-6557b66dbd-6224h" event={"ID":"570ab3a3-e08f-47b8-8273-49852b16ce07","Type":"ContainerStarted","Data":"79e2f12279036f07f2be83a0969b0d2307e021940c9ae56c3e545dcce707a895"}
Jul 07 11:37:13.293799 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:37:13.293607    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-8bb53107-6557b66dbd-6224h" event={"ID":"570ab3a3-e08f-47b8-8273-49852b16ce07","Type":"ContainerStarted","Data":"271aafdf15a5cfa95917ebdad3ccfa921a4bb0bba8a1922ba203c0db692e024e"}
Jul 07 11:37:13.293799 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:37:13.293671    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-derive-8bb53107-6557b66dbd-6224h"
Jul 07 11:37:16.312964 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:37:16.312913    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-derive-8bb53107-6557b66dbd-6224h" podStartSLOduration=4.971336689 podStartE2EDuration="5.312896471s" totalImagesPullingTime="341.559782ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:37:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:37:13.329573011 +0000 UTC m=+1583.039469398" watchObservedRunningTime="2026-07-07 11:37:16.312896471 +0000 UTC m=+1586.022792858"
Jul 07 11:37:16.313622 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:37:16.313606    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-8bb53107-6557b66dbd-6224h"]
Jul 07 11:37:16.313792 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:37:16.313769    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-derive-8bb53107-6557b66dbd-6224h" podUID="570ab3a3-e08f-47b8-8273-49852b16ce07" containerName="maas-api" containerID="cri-o://79e2f12279036f07f2be83a0969b0d2307e021940c9ae56c3e545dcce707a895" gracePeriod=30
Jul 07 11:37:16.570954 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:37:16.570881    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-8bb53107-6557b66dbd-6224h"
Jul 07 11:37:16.723221 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:37:16.723188    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qxx6v\" (UniqueName: \"kubernetes.io/projected/570ab3a3-e08f-47b8-8273-49852b16ce07-kube-api-access-qxx6v\") pod \"570ab3a3-e08f-47b8-8273-49852b16ce07\" (UID: \"570ab3a3-e08f-47b8-8273-49852b16ce07\") "
Jul 07 11:37:16.723433 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:37:16.723239    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/570ab3a3-e08f-47b8-8273-49852b16ce07-openshift-service-ca\") pod \"570ab3a3-e08f-47b8-8273-49852b16ce07\" (UID: \"570ab3a3-e08f-47b8-8273-49852b16ce07\") "
Jul 07 11:37:16.723433 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:37:16.723270    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/570ab3a3-e08f-47b8-8273-49852b16ce07-maas-api-tls\") pod \"570ab3a3-e08f-47b8-8273-49852b16ce07\" (UID: \"570ab3a3-e08f-47b8-8273-49852b16ce07\") "
Jul 07 11:37:16.723631 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:37:16.723607    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/570ab3a3-e08f-47b8-8273-49852b16ce07-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "570ab3a3-e08f-47b8-8273-49852b16ce07" (UID: "570ab3a3-e08f-47b8-8273-49852b16ce07"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 11:37:16.725525 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:37:16.725504    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/570ab3a3-e08f-47b8-8273-49852b16ce07-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "570ab3a3-e08f-47b8-8273-49852b16ce07" (UID: "570ab3a3-e08f-47b8-8273-49852b16ce07"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 11:37:16.725744 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:37:16.725719    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/570ab3a3-e08f-47b8-8273-49852b16ce07-kube-api-access-qxx6v" (OuterVolumeSpecName: "kube-api-access-qxx6v") pod "570ab3a3-e08f-47b8-8273-49852b16ce07" (UID: "570ab3a3-e08f-47b8-8273-49852b16ce07"). InnerVolumeSpecName "kube-api-access-qxx6v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:37:16.824096 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:37:16.824017    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxx6v\" (UniqueName: \"kubernetes.io/projected/570ab3a3-e08f-47b8-8273-49852b16ce07-kube-api-access-qxx6v\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:37:16.824096 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:37:16.824043    2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/570ab3a3-e08f-47b8-8273-49852b16ce07-openshift-service-ca\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:37:16.824096 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:37:16.824054    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/570ab3a3-e08f-47b8-8273-49852b16ce07-maas-api-tls\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:37:17.315320 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:37:17.315286    2570 generic.go:356] "Generic (PLEG): container finished" podID="570ab3a3-e08f-47b8-8273-49852b16ce07" containerID="79e2f12279036f07f2be83a0969b0d2307e021940c9ae56c3e545dcce707a895" exitCode=0
Jul 07 11:37:17.315803 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:37:17.315366    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-8bb53107-6557b66dbd-6224h"
Jul 07 11:37:17.315803 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:37:17.315375    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-8bb53107-6557b66dbd-6224h" event={"ID":"570ab3a3-e08f-47b8-8273-49852b16ce07","Type":"ContainerDied","Data":"79e2f12279036f07f2be83a0969b0d2307e021940c9ae56c3e545dcce707a895"}
Jul 07 11:37:17.315803 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:37:17.315439    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-8bb53107-6557b66dbd-6224h" event={"ID":"570ab3a3-e08f-47b8-8273-49852b16ce07","Type":"ContainerDied","Data":"271aafdf15a5cfa95917ebdad3ccfa921a4bb0bba8a1922ba203c0db692e024e"}
Jul 07 11:37:17.315803 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:37:17.315464    2570 scope.go:117] "RemoveContainer" containerID="79e2f12279036f07f2be83a0969b0d2307e021940c9ae56c3e545dcce707a895"
Jul 07 11:37:17.326491 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:37:17.326472    2570 scope.go:117] "RemoveContainer" containerID="79e2f12279036f07f2be83a0969b0d2307e021940c9ae56c3e545dcce707a895"
Jul 07 11:37:17.326825 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:37:17.326800    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79e2f12279036f07f2be83a0969b0d2307e021940c9ae56c3e545dcce707a895\": container with ID starting with 79e2f12279036f07f2be83a0969b0d2307e021940c9ae56c3e545dcce707a895 not found: ID does not exist" containerID="79e2f12279036f07f2be83a0969b0d2307e021940c9ae56c3e545dcce707a895"
Jul 07 11:37:17.326877 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:37:17.326836    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79e2f12279036f07f2be83a0969b0d2307e021940c9ae56c3e545dcce707a895"} err="failed to get container status \"79e2f12279036f07f2be83a0969b0d2307e021940c9ae56c3e545dcce707a895\": rpc error: code = NotFound desc = could not find container \"79e2f12279036f07f2be83a0969b0d2307e021940c9ae56c3e545dcce707a895\": container with ID starting with 79e2f12279036f07f2be83a0969b0d2307e021940c9ae56c3e545dcce707a895 not found: ID does not exist"
Jul 07 11:37:17.352662 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:37:17.352628    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-8bb53107-6557b66dbd-6224h"]
Jul 07 11:37:17.357294 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:37:17.357267    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-derive-8bb53107-6557b66dbd-6224h"]
Jul 07 11:37:18.820305 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:37:18.820275    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="570ab3a3-e08f-47b8-8273-49852b16ce07" path="/var/lib/kubelet/pods/570ab3a3-e08f-47b8-8273-49852b16ce07/volumes"
Jul 07 11:39:11.765311 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:11.765238    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-ce6aaeca-58fb556669-xlf25"]
Jul 07 11:39:11.765782 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:11.765740    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="570ab3a3-e08f-47b8-8273-49852b16ce07" containerName="maas-api"
Jul 07 11:39:11.765782 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:11.765757    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="570ab3a3-e08f-47b8-8273-49852b16ce07" containerName="maas-api"
Jul 07 11:39:11.765854 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:11.765836    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="570ab3a3-e08f-47b8-8273-49852b16ce07" containerName="maas-api"
Jul 07 11:39:11.768955 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:11.768934    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-ce6aaeca-58fb556669-xlf25"
Jul 07 11:39:11.771858 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:11.771836    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-ce6aaeca-serving-cert\""
Jul 07 11:39:11.781839 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:11.781812    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-ce6aaeca-58fb556669-xlf25"]
Jul 07 11:39:11.905939 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:11.905882    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1db80a64-67e8-492b-a8d9-ef8891c44d6d-openshift-service-ca\") pod \"maas-api-e2e-mt-ce6aaeca-58fb556669-xlf25\" (UID: \"1db80a64-67e8-492b-a8d9-ef8891c44d6d\") " pod="opendatahub/maas-api-e2e-mt-ce6aaeca-58fb556669-xlf25"
Jul 07 11:39:11.905939 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:11.905940    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9zw6\" (UniqueName: \"kubernetes.io/projected/1db80a64-67e8-492b-a8d9-ef8891c44d6d-kube-api-access-v9zw6\") pod \"maas-api-e2e-mt-ce6aaeca-58fb556669-xlf25\" (UID: \"1db80a64-67e8-492b-a8d9-ef8891c44d6d\") " pod="opendatahub/maas-api-e2e-mt-ce6aaeca-58fb556669-xlf25"
Jul 07 11:39:11.906228 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:11.905992    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1db80a64-67e8-492b-a8d9-ef8891c44d6d-maas-api-tls\") pod \"maas-api-e2e-mt-ce6aaeca-58fb556669-xlf25\" (UID: \"1db80a64-67e8-492b-a8d9-ef8891c44d6d\") " pod="opendatahub/maas-api-e2e-mt-ce6aaeca-58fb556669-xlf25"
Jul 07 11:39:12.006752 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:12.006716    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1db80a64-67e8-492b-a8d9-ef8891c44d6d-openshift-service-ca\") pod \"maas-api-e2e-mt-ce6aaeca-58fb556669-xlf25\" (UID: \"1db80a64-67e8-492b-a8d9-ef8891c44d6d\") " pod="opendatahub/maas-api-e2e-mt-ce6aaeca-58fb556669-xlf25"
Jul 07 11:39:12.006752 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:12.006752    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9zw6\" (UniqueName: \"kubernetes.io/projected/1db80a64-67e8-492b-a8d9-ef8891c44d6d-kube-api-access-v9zw6\") pod \"maas-api-e2e-mt-ce6aaeca-58fb556669-xlf25\" (UID: \"1db80a64-67e8-492b-a8d9-ef8891c44d6d\") " pod="opendatahub/maas-api-e2e-mt-ce6aaeca-58fb556669-xlf25"
Jul 07 11:39:12.006978 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:12.006792    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1db80a64-67e8-492b-a8d9-ef8891c44d6d-maas-api-tls\") pod \"maas-api-e2e-mt-ce6aaeca-58fb556669-xlf25\" (UID: \"1db80a64-67e8-492b-a8d9-ef8891c44d6d\") " pod="opendatahub/maas-api-e2e-mt-ce6aaeca-58fb556669-xlf25"
Jul 07 11:39:12.007365 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:12.007345    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1db80a64-67e8-492b-a8d9-ef8891c44d6d-openshift-service-ca\") pod \"maas-api-e2e-mt-ce6aaeca-58fb556669-xlf25\" (UID: \"1db80a64-67e8-492b-a8d9-ef8891c44d6d\") " pod="opendatahub/maas-api-e2e-mt-ce6aaeca-58fb556669-xlf25"
Jul 07 11:39:12.009386 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:12.009356    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1db80a64-67e8-492b-a8d9-ef8891c44d6d-maas-api-tls\") pod \"maas-api-e2e-mt-ce6aaeca-58fb556669-xlf25\" (UID: \"1db80a64-67e8-492b-a8d9-ef8891c44d6d\") " pod="opendatahub/maas-api-e2e-mt-ce6aaeca-58fb556669-xlf25"
Jul 07 11:39:12.017290 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:12.017228    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9zw6\" (UniqueName: \"kubernetes.io/projected/1db80a64-67e8-492b-a8d9-ef8891c44d6d-kube-api-access-v9zw6\") pod \"maas-api-e2e-mt-ce6aaeca-58fb556669-xlf25\" (UID: \"1db80a64-67e8-492b-a8d9-ef8891c44d6d\") " pod="opendatahub/maas-api-e2e-mt-ce6aaeca-58fb556669-xlf25"
Jul 07 11:39:12.080344 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:12.080302    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-ce6aaeca-58fb556669-xlf25"
Jul 07 11:39:12.222397 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:12.222373    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-ce6aaeca-58fb556669-xlf25"]
Jul 07 11:39:12.225036 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:39:12.225008    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1db80a64_67e8_492b_a8d9_ef8891c44d6d.slice/crio-d35341b5ba8e974b518c2a04755dda54013a5adb4d0638f3ece07661c6b904d6 WatchSource:0}: Error finding container d35341b5ba8e974b518c2a04755dda54013a5adb4d0638f3ece07661c6b904d6: Status 404 returned error can't find the container with id d35341b5ba8e974b518c2a04755dda54013a5adb4d0638f3ece07661c6b904d6
Jul 07 11:39:12.873169 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:12.873081    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-ce6aaeca-58fb556669-xlf25" event={"ID":"1db80a64-67e8-492b-a8d9-ef8891c44d6d","Type":"ContainerStarted","Data":"da204df95494cc02d6a538b34d3e8d65786479d1163ba15b10050304a4280184"}
Jul 07 11:39:12.873663 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:12.873191    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-ce6aaeca-58fb556669-xlf25" event={"ID":"1db80a64-67e8-492b-a8d9-ef8891c44d6d","Type":"ContainerStarted","Data":"d35341b5ba8e974b518c2a04755dda54013a5adb4d0638f3ece07661c6b904d6"}
Jul 07 11:39:12.873663 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:12.873220    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-ce6aaeca-58fb556669-xlf25"
Jul 07 11:39:12.893105 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:12.893059    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-ce6aaeca-58fb556669-xlf25" podStartSLOduration=1.462660469 podStartE2EDuration="1.893043009s" totalImagesPullingTime="430.38254ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:39:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:39:12.891949252 +0000 UTC m=+1702.601845638" watchObservedRunningTime="2026-07-07 11:39:12.893043009 +0000 UTC m=+1702.602939397"
Jul 07 11:39:18.883875 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:18.883840    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-ce6aaeca-58fb556669-xlf25"
Jul 07 11:39:24.086832 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:24.086797    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-ce6aaeca-58fb556669-xlf25"]
Jul 07 11:39:24.087314 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:24.087102    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-ce6aaeca-58fb556669-xlf25" podUID="1db80a64-67e8-492b-a8d9-ef8891c44d6d" containerName="maas-api" containerID="cri-o://da204df95494cc02d6a538b34d3e8d65786479d1163ba15b10050304a4280184" gracePeriod=30
Jul 07 11:39:24.346169 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:24.346109    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-ce6aaeca-58fb556669-xlf25"
Jul 07 11:39:24.417900 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:24.417862    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1db80a64-67e8-492b-a8d9-ef8891c44d6d-openshift-service-ca\") pod \"1db80a64-67e8-492b-a8d9-ef8891c44d6d\" (UID: \"1db80a64-67e8-492b-a8d9-ef8891c44d6d\") "
Jul 07 11:39:24.418069 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:24.417963    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v9zw6\" (UniqueName: \"kubernetes.io/projected/1db80a64-67e8-492b-a8d9-ef8891c44d6d-kube-api-access-v9zw6\") pod \"1db80a64-67e8-492b-a8d9-ef8891c44d6d\" (UID: \"1db80a64-67e8-492b-a8d9-ef8891c44d6d\") "
Jul 07 11:39:24.418069 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:24.417989    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1db80a64-67e8-492b-a8d9-ef8891c44d6d-maas-api-tls\") pod \"1db80a64-67e8-492b-a8d9-ef8891c44d6d\" (UID: \"1db80a64-67e8-492b-a8d9-ef8891c44d6d\") "
Jul 07 11:39:24.418289 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:24.418265    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1db80a64-67e8-492b-a8d9-ef8891c44d6d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1db80a64-67e8-492b-a8d9-ef8891c44d6d" (UID: "1db80a64-67e8-492b-a8d9-ef8891c44d6d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 11:39:24.420276 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:24.420243    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1db80a64-67e8-492b-a8d9-ef8891c44d6d-kube-api-access-v9zw6" (OuterVolumeSpecName: "kube-api-access-v9zw6") pod "1db80a64-67e8-492b-a8d9-ef8891c44d6d" (UID: "1db80a64-67e8-492b-a8d9-ef8891c44d6d"). InnerVolumeSpecName "kube-api-access-v9zw6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:39:24.420406 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:24.420241    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1db80a64-67e8-492b-a8d9-ef8891c44d6d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1db80a64-67e8-492b-a8d9-ef8891c44d6d" (UID: "1db80a64-67e8-492b-a8d9-ef8891c44d6d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 11:39:24.518967 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:24.518932    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v9zw6\" (UniqueName: \"kubernetes.io/projected/1db80a64-67e8-492b-a8d9-ef8891c44d6d-kube-api-access-v9zw6\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:39:24.518967 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:24.518963    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1db80a64-67e8-492b-a8d9-ef8891c44d6d-maas-api-tls\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:39:24.518967 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:24.518973    2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1db80a64-67e8-492b-a8d9-ef8891c44d6d-openshift-service-ca\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:39:24.933762 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:24.933728    2570 generic.go:356] "Generic (PLEG): container finished" podID="1db80a64-67e8-492b-a8d9-ef8891c44d6d" containerID="da204df95494cc02d6a538b34d3e8d65786479d1163ba15b10050304a4280184" exitCode=0
Jul 07 11:39:24.933960 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:24.933779    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-ce6aaeca-58fb556669-xlf25" event={"ID":"1db80a64-67e8-492b-a8d9-ef8891c44d6d","Type":"ContainerDied","Data":"da204df95494cc02d6a538b34d3e8d65786479d1163ba15b10050304a4280184"}
Jul 07 11:39:24.933960 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:24.933785    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-ce6aaeca-58fb556669-xlf25"
Jul 07 11:39:24.933960 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:24.933821    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-ce6aaeca-58fb556669-xlf25" event={"ID":"1db80a64-67e8-492b-a8d9-ef8891c44d6d","Type":"ContainerDied","Data":"d35341b5ba8e974b518c2a04755dda54013a5adb4d0638f3ece07661c6b904d6"}
Jul 07 11:39:24.933960 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:24.933842    2570 scope.go:117] "RemoveContainer" containerID="da204df95494cc02d6a538b34d3e8d65786479d1163ba15b10050304a4280184"
Jul 07 11:39:24.942607 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:24.942588    2570 scope.go:117] "RemoveContainer" containerID="da204df95494cc02d6a538b34d3e8d65786479d1163ba15b10050304a4280184"
Jul 07 11:39:24.942844 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:39:24.942823    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da204df95494cc02d6a538b34d3e8d65786479d1163ba15b10050304a4280184\": container with ID starting with da204df95494cc02d6a538b34d3e8d65786479d1163ba15b10050304a4280184 not found: ID does not exist" containerID="da204df95494cc02d6a538b34d3e8d65786479d1163ba15b10050304a4280184"
Jul 07 11:39:24.942936 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:24.942849    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da204df95494cc02d6a538b34d3e8d65786479d1163ba15b10050304a4280184"} err="failed to get container status \"da204df95494cc02d6a538b34d3e8d65786479d1163ba15b10050304a4280184\": rpc error: code = NotFound desc = could not find container \"da204df95494cc02d6a538b34d3e8d65786479d1163ba15b10050304a4280184\": container with ID starting with da204df95494cc02d6a538b34d3e8d65786479d1163ba15b10050304a4280184 not found: ID does not exist"
Jul 07 11:39:24.952261 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:24.952239    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-ce6aaeca-58fb556669-xlf25"]
Jul 07 11:39:24.956219 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:24.956195    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-ce6aaeca-58fb556669-xlf25"]
Jul 07 11:39:26.820956 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:26.820926    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1db80a64-67e8-492b-a8d9-ef8891c44d6d" path="/var/lib/kubelet/pods/1db80a64-67e8-492b-a8d9-ef8891c44d6d/volumes"
Jul 07 11:39:52.965500 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:52.965470    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-180e9350-66f7cb65cc-pdgkr"]
Jul 07 11:39:52.965920 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:52.965841    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1db80a64-67e8-492b-a8d9-ef8891c44d6d" containerName="maas-api"
Jul 07 11:39:52.965920 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:52.965851    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="1db80a64-67e8-492b-a8d9-ef8891c44d6d" containerName="maas-api"
Jul 07 11:39:52.965998 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:52.965926    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="1db80a64-67e8-492b-a8d9-ef8891c44d6d" containerName="maas-api"
Jul 07 11:39:52.969051 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:52.969033    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-180e9350-66f7cb65cc-pdgkr"
Jul 07 11:39:52.972668 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:52.972643    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-180e9350-serving-cert\""
Jul 07 11:39:52.980928 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:52.980903    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-180e9350-66f7cb65cc-pdgkr"]
Jul 07 11:39:53.086011 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:53.085974    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtwmj\" (UniqueName: \"kubernetes.io/projected/50d956eb-997c-4139-a37c-a2ca924fe391-kube-api-access-jtwmj\") pod \"maas-api-e2e-mt-180e9350-66f7cb65cc-pdgkr\" (UID: \"50d956eb-997c-4139-a37c-a2ca924fe391\") " pod="opendatahub/maas-api-e2e-mt-180e9350-66f7cb65cc-pdgkr"
Jul 07 11:39:53.086200 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:53.086030    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50d956eb-997c-4139-a37c-a2ca924fe391-openshift-service-ca\") pod \"maas-api-e2e-mt-180e9350-66f7cb65cc-pdgkr\" (UID: \"50d956eb-997c-4139-a37c-a2ca924fe391\") " pod="opendatahub/maas-api-e2e-mt-180e9350-66f7cb65cc-pdgkr"
Jul 07 11:39:53.086200 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:53.086112    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50d956eb-997c-4139-a37c-a2ca924fe391-maas-api-tls\") pod \"maas-api-e2e-mt-180e9350-66f7cb65cc-pdgkr\" (UID: \"50d956eb-997c-4139-a37c-a2ca924fe391\") " pod="opendatahub/maas-api-e2e-mt-180e9350-66f7cb65cc-pdgkr"
Jul 07 11:39:53.186752 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:53.186715    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50d956eb-997c-4139-a37c-a2ca924fe391-maas-api-tls\") pod \"maas-api-e2e-mt-180e9350-66f7cb65cc-pdgkr\" (UID: \"50d956eb-997c-4139-a37c-a2ca924fe391\") " pod="opendatahub/maas-api-e2e-mt-180e9350-66f7cb65cc-pdgkr"
Jul 07 11:39:53.186925 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:53.186836    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtwmj\" (UniqueName: \"kubernetes.io/projected/50d956eb-997c-4139-a37c-a2ca924fe391-kube-api-access-jtwmj\") pod \"maas-api-e2e-mt-180e9350-66f7cb65cc-pdgkr\" (UID: \"50d956eb-997c-4139-a37c-a2ca924fe391\") " pod="opendatahub/maas-api-e2e-mt-180e9350-66f7cb65cc-pdgkr"
Jul 07 11:39:53.186925 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:53.186877    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50d956eb-997c-4139-a37c-a2ca924fe391-openshift-service-ca\") pod \"maas-api-e2e-mt-180e9350-66f7cb65cc-pdgkr\" (UID: \"50d956eb-997c-4139-a37c-a2ca924fe391\") " pod="opendatahub/maas-api-e2e-mt-180e9350-66f7cb65cc-pdgkr"
Jul 07 11:39:53.187575 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:53.187556    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50d956eb-997c-4139-a37c-a2ca924fe391-openshift-service-ca\") pod \"maas-api-e2e-mt-180e9350-66f7cb65cc-pdgkr\" (UID: \"50d956eb-997c-4139-a37c-a2ca924fe391\") " pod="opendatahub/maas-api-e2e-mt-180e9350-66f7cb65cc-pdgkr"
Jul 07 11:39:53.189518 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:53.189498    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50d956eb-997c-4139-a37c-a2ca924fe391-maas-api-tls\") pod \"maas-api-e2e-mt-180e9350-66f7cb65cc-pdgkr\" (UID: \"50d956eb-997c-4139-a37c-a2ca924fe391\") " pod="opendatahub/maas-api-e2e-mt-180e9350-66f7cb65cc-pdgkr"
Jul 07 11:39:53.197443 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:53.197414    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtwmj\" (UniqueName: \"kubernetes.io/projected/50d956eb-997c-4139-a37c-a2ca924fe391-kube-api-access-jtwmj\") pod \"maas-api-e2e-mt-180e9350-66f7cb65cc-pdgkr\" (UID: \"50d956eb-997c-4139-a37c-a2ca924fe391\") " pod="opendatahub/maas-api-e2e-mt-180e9350-66f7cb65cc-pdgkr"
Jul 07 11:39:53.281277 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:53.281243    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-180e9350-66f7cb65cc-pdgkr"
Jul 07 11:39:53.431354 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:53.431304    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-180e9350-66f7cb65cc-pdgkr"]
Jul 07 11:39:53.433459 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:39:53.433433    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50d956eb_997c_4139_a37c_a2ca924fe391.slice/crio-15e7772f180af4d3640fe5185d2be182bd661b5f321ff24fe13dd3f39e613340 WatchSource:0}: Error finding container 15e7772f180af4d3640fe5185d2be182bd661b5f321ff24fe13dd3f39e613340: Status 404 returned error can't find the container with id 15e7772f180af4d3640fe5185d2be182bd661b5f321ff24fe13dd3f39e613340
Jul 07 11:39:54.095660 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:54.095621    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-180e9350-66f7cb65cc-pdgkr" event={"ID":"50d956eb-997c-4139-a37c-a2ca924fe391","Type":"ContainerStarted","Data":"91295946411db4d3a64f25dd9f4f0a0ef969f07ecaf5b07c0231d8dc80d30ce1"}
Jul 07 11:39:54.096088 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:54.095673    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-180e9350-66f7cb65cc-pdgkr" event={"ID":"50d956eb-997c-4139-a37c-a2ca924fe391","Type":"ContainerStarted","Data":"15e7772f180af4d3640fe5185d2be182bd661b5f321ff24fe13dd3f39e613340"}
Jul 07 11:39:54.096088 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:54.095773    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-180e9350-66f7cb65cc-pdgkr"
Jul 07 11:39:54.124731 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:39:54.124685    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-180e9350-66f7cb65cc-pdgkr" podStartSLOduration=1.587822756 podStartE2EDuration="2.124667155s" totalImagesPullingTime="536.844399ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:39:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:39:54.123092878 +0000 UTC m=+1743.832989268" watchObservedRunningTime="2026-07-07 11:39:54.124667155 +0000 UTC m=+1743.834563588"
Jul 07 11:40:00.105778 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:00.105745    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-180e9350-66f7cb65cc-pdgkr"
Jul 07 11:40:11.123894 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:11.123854    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-180e9350/e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm"]
Jul 07 11:40:11.127601 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:11.127580    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-180e9350/e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm"
Jul 07 11:40:11.132498 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:11.132469    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-180e9350\"/\"default-dockercfg-kkdcm\""
Jul 07 11:40:11.132638 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:11.132619    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-180e9350\"/\"openshift-service-ca.crt\""
Jul 07 11:40:11.132721 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:11.132630    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-180e9350\"/\"e2e-lifecycle-model-180e9350-kserve-self-signed-certs\""
Jul 07 11:40:11.132721 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:11.132640    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-180e9350\"/\"kube-root-ca.crt\""
Jul 07 11:40:11.140211 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:11.140189    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-180e9350/e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm"]
Jul 07 11:40:11.146651 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:11.146621    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a165b186-347d-4bf9-900a-0c86ddd1fa5a-tmp-dir\") pod \"e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm\" (UID: \"a165b186-347d-4bf9-900a-0c86ddd1fa5a\") " pod="ai-tenant-e2e-mt-180e9350/e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm"
Jul 07 11:40:11.146780 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:11.146675    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a165b186-347d-4bf9-900a-0c86ddd1fa5a-home\") pod \"e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm\" (UID: \"a165b186-347d-4bf9-900a-0c86ddd1fa5a\") " pod="ai-tenant-e2e-mt-180e9350/e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm"
Jul 07 11:40:11.146878 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:11.146808    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9d9k\" (UniqueName: \"kubernetes.io/projected/a165b186-347d-4bf9-900a-0c86ddd1fa5a-kube-api-access-h9d9k\") pod \"e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm\" (UID: \"a165b186-347d-4bf9-900a-0c86ddd1fa5a\") " pod="ai-tenant-e2e-mt-180e9350/e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm"
Jul 07 11:40:11.147041 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:11.147019    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a165b186-347d-4bf9-900a-0c86ddd1fa5a-model-cache\") pod \"e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm\" (UID: \"a165b186-347d-4bf9-900a-0c86ddd1fa5a\") " pod="ai-tenant-e2e-mt-180e9350/e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm"
Jul 07 11:40:11.147129 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:11.147084    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a165b186-347d-4bf9-900a-0c86ddd1fa5a-dshm\") pod \"e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm\" (UID: \"a165b186-347d-4bf9-900a-0c86ddd1fa5a\") " pod="ai-tenant-e2e-mt-180e9350/e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm"
Jul 07 11:40:11.147129 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:11.147113    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a165b186-347d-4bf9-900a-0c86ddd1fa5a-tls-certs\") pod \"e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm\" (UID: \"a165b186-347d-4bf9-900a-0c86ddd1fa5a\") " pod="ai-tenant-e2e-mt-180e9350/e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm"
Jul 07 11:40:11.248171 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:11.248137    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a165b186-347d-4bf9-900a-0c86ddd1fa5a-tmp-dir\") pod \"e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm\" (UID: \"a165b186-347d-4bf9-900a-0c86ddd1fa5a\") " pod="ai-tenant-e2e-mt-180e9350/e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm"
Jul 07 11:40:11.248377 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:11.248178    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a165b186-347d-4bf9-900a-0c86ddd1fa5a-home\") pod \"e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm\" (UID: \"a165b186-347d-4bf9-900a-0c86ddd1fa5a\") " pod="ai-tenant-e2e-mt-180e9350/e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm"
Jul 07 11:40:11.248377 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:11.248203    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9d9k\" (UniqueName: \"kubernetes.io/projected/a165b186-347d-4bf9-900a-0c86ddd1fa5a-kube-api-access-h9d9k\") pod \"e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm\" (UID: \"a165b186-347d-4bf9-900a-0c86ddd1fa5a\") " pod="ai-tenant-e2e-mt-180e9350/e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm"
Jul 07 11:40:11.248377 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:11.248259    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a165b186-347d-4bf9-900a-0c86ddd1fa5a-model-cache\") pod \"e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm\" (UID: \"a165b186-347d-4bf9-900a-0c86ddd1fa5a\") " pod="ai-tenant-e2e-mt-180e9350/e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm"
Jul 07 11:40:11.248377 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:11.248302    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a165b186-347d-4bf9-900a-0c86ddd1fa5a-dshm\") pod \"e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm\" (UID: \"a165b186-347d-4bf9-900a-0c86ddd1fa5a\") " pod="ai-tenant-e2e-mt-180e9350/e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm"
Jul 07 11:40:11.248377 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:11.248358    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a165b186-347d-4bf9-900a-0c86ddd1fa5a-tls-certs\") pod \"e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm\" (UID: \"a165b186-347d-4bf9-900a-0c86ddd1fa5a\") " pod="ai-tenant-e2e-mt-180e9350/e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm"
Jul 07 11:40:11.248637 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:11.248578    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a165b186-347d-4bf9-900a-0c86ddd1fa5a-home\") pod \"e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm\" (UID: \"a165b186-347d-4bf9-900a-0c86ddd1fa5a\") " pod="ai-tenant-e2e-mt-180e9350/e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm"
Jul 07 11:40:11.248696 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:11.248636    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a165b186-347d-4bf9-900a-0c86ddd1fa5a-tmp-dir\") pod \"e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm\" (UID: \"a165b186-347d-4bf9-900a-0c86ddd1fa5a\") " pod="ai-tenant-e2e-mt-180e9350/e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm"
Jul 07 11:40:11.248761 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:11.248743    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a165b186-347d-4bf9-900a-0c86ddd1fa5a-model-cache\") pod \"e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm\" (UID: \"a165b186-347d-4bf9-900a-0c86ddd1fa5a\") " pod="ai-tenant-e2e-mt-180e9350/e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm"
Jul 07 11:40:11.250702 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:11.250681    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a165b186-347d-4bf9-900a-0c86ddd1fa5a-dshm\") pod \"e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm\" (UID: \"a165b186-347d-4bf9-900a-0c86ddd1fa5a\") " pod="ai-tenant-e2e-mt-180e9350/e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm"
Jul 07 11:40:11.250889 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:11.250871    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a165b186-347d-4bf9-900a-0c86ddd1fa5a-tls-certs\") pod \"e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm\" (UID: \"a165b186-347d-4bf9-900a-0c86ddd1fa5a\") " pod="ai-tenant-e2e-mt-180e9350/e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm"
Jul 07 11:40:11.258025 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:11.257991    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9d9k\" (UniqueName: \"kubernetes.io/projected/a165b186-347d-4bf9-900a-0c86ddd1fa5a-kube-api-access-h9d9k\") pod \"e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm\" (UID: \"a165b186-347d-4bf9-900a-0c86ddd1fa5a\") " pod="ai-tenant-e2e-mt-180e9350/e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm"
Jul 07 11:40:11.437465 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:11.437368    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-180e9350/e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm"
Jul 07 11:40:11.575671 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:11.575647    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-180e9350/e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm"]
Jul 07 11:40:11.577299 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:40:11.577273    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda165b186_347d_4bf9_900a_0c86ddd1fa5a.slice/crio-d6c11caf9e80ab2c141de50b8df15b47740be5dd64675e52c4630c60101e48c9 WatchSource:0}: Error finding container d6c11caf9e80ab2c141de50b8df15b47740be5dd64675e52c4630c60101e48c9: Status 404 returned error can't find the container with id d6c11caf9e80ab2c141de50b8df15b47740be5dd64675e52c4630c60101e48c9
Jul 07 11:40:12.184978 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:12.184935    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-180e9350/e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm" event={"ID":"a165b186-347d-4bf9-900a-0c86ddd1fa5a","Type":"ContainerStarted","Data":"abbf5c536b75fc68718a1fa358ac66313f518ceee2db712672df839d7ff18414"}
Jul 07 11:40:12.185395 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:12.184994    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-180e9350/e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm" event={"ID":"a165b186-347d-4bf9-900a-0c86ddd1fa5a","Type":"ContainerStarted","Data":"d6c11caf9e80ab2c141de50b8df15b47740be5dd64675e52c4630c60101e48c9"}
Jul 07 11:40:12.209025 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:12.208970    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-180e9350/e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm" podStartSLOduration=1.208954802 podStartE2EDuration="1.208954802s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:40:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:40:12.206212928 +0000 UTC m=+1761.916109316" watchObservedRunningTime="2026-07-07 11:40:12.208954802 +0000 UTC m=+1761.918851189"
Jul 07 11:40:21.437821 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:21.437775    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-180e9350/e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm"
Jul 07 11:40:21.438256 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:21.437854    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-180e9350/e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm"
Jul 07 11:40:21.450538 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:21.450508    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-180e9350/e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm"
Jul 07 11:40:22.242214 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:22.242186    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-180e9350/e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm"
Jul 07 11:40:40.494096 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:40.494054    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-180e9350-66f7cb65cc-pdgkr"]
Jul 07 11:40:40.494585 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:40.494319    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-180e9350-66f7cb65cc-pdgkr" podUID="50d956eb-997c-4139-a37c-a2ca924fe391" containerName="maas-api" containerID="cri-o://91295946411db4d3a64f25dd9f4f0a0ef969f07ecaf5b07c0231d8dc80d30ce1" gracePeriod=30
Jul 07 11:40:40.746103 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:40.746039    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-180e9350-66f7cb65cc-pdgkr"
Jul 07 11:40:40.819055 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:40.819028    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jtwmj\" (UniqueName: \"kubernetes.io/projected/50d956eb-997c-4139-a37c-a2ca924fe391-kube-api-access-jtwmj\") pod \"50d956eb-997c-4139-a37c-a2ca924fe391\" (UID: \"50d956eb-997c-4139-a37c-a2ca924fe391\") "
Jul 07 11:40:40.819250 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:40.819142    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50d956eb-997c-4139-a37c-a2ca924fe391-maas-api-tls\") pod \"50d956eb-997c-4139-a37c-a2ca924fe391\" (UID: \"50d956eb-997c-4139-a37c-a2ca924fe391\") "
Jul 07 11:40:40.819250 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:40.819199    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50d956eb-997c-4139-a37c-a2ca924fe391-openshift-service-ca\") pod \"50d956eb-997c-4139-a37c-a2ca924fe391\" (UID: \"50d956eb-997c-4139-a37c-a2ca924fe391\") "
Jul 07 11:40:40.819661 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:40.819607    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50d956eb-997c-4139-a37c-a2ca924fe391-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "50d956eb-997c-4139-a37c-a2ca924fe391" (UID: "50d956eb-997c-4139-a37c-a2ca924fe391"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 11:40:40.821402 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:40.821375    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50d956eb-997c-4139-a37c-a2ca924fe391-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "50d956eb-997c-4139-a37c-a2ca924fe391" (UID: "50d956eb-997c-4139-a37c-a2ca924fe391"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 11:40:40.821497 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:40.821428    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50d956eb-997c-4139-a37c-a2ca924fe391-kube-api-access-jtwmj" (OuterVolumeSpecName: "kube-api-access-jtwmj") pod "50d956eb-997c-4139-a37c-a2ca924fe391" (UID: "50d956eb-997c-4139-a37c-a2ca924fe391"). InnerVolumeSpecName "kube-api-access-jtwmj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:40:40.920778 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:40.920737    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50d956eb-997c-4139-a37c-a2ca924fe391-maas-api-tls\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:40:40.920778 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:40.920773    2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50d956eb-997c-4139-a37c-a2ca924fe391-openshift-service-ca\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:40:40.920778 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:40.920784    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jtwmj\" (UniqueName: \"kubernetes.io/projected/50d956eb-997c-4139-a37c-a2ca924fe391-kube-api-access-jtwmj\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:40:41.324517 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:41.324482    2570 generic.go:356] "Generic (PLEG): container finished" podID="50d956eb-997c-4139-a37c-a2ca924fe391" containerID="91295946411db4d3a64f25dd9f4f0a0ef969f07ecaf5b07c0231d8dc80d30ce1" exitCode=0
Jul 07 11:40:41.324699 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:41.324527    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-180e9350-66f7cb65cc-pdgkr"
Jul 07 11:40:41.324699 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:41.324570    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-180e9350-66f7cb65cc-pdgkr" event={"ID":"50d956eb-997c-4139-a37c-a2ca924fe391","Type":"ContainerDied","Data":"91295946411db4d3a64f25dd9f4f0a0ef969f07ecaf5b07c0231d8dc80d30ce1"}
Jul 07 11:40:41.324699 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:41.324628    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-180e9350-66f7cb65cc-pdgkr" event={"ID":"50d956eb-997c-4139-a37c-a2ca924fe391","Type":"ContainerDied","Data":"15e7772f180af4d3640fe5185d2be182bd661b5f321ff24fe13dd3f39e613340"}
Jul 07 11:40:41.324699 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:41.324652    2570 scope.go:117] "RemoveContainer" containerID="91295946411db4d3a64f25dd9f4f0a0ef969f07ecaf5b07c0231d8dc80d30ce1"
Jul 07 11:40:41.333550 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:41.333534    2570 scope.go:117] "RemoveContainer" containerID="91295946411db4d3a64f25dd9f4f0a0ef969f07ecaf5b07c0231d8dc80d30ce1"
Jul 07 11:40:41.333810 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:40:41.333792    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91295946411db4d3a64f25dd9f4f0a0ef969f07ecaf5b07c0231d8dc80d30ce1\": container with ID starting with 91295946411db4d3a64f25dd9f4f0a0ef969f07ecaf5b07c0231d8dc80d30ce1 not found: ID does not exist" containerID="91295946411db4d3a64f25dd9f4f0a0ef969f07ecaf5b07c0231d8dc80d30ce1"
Jul 07 11:40:41.333858 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:41.333819    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91295946411db4d3a64f25dd9f4f0a0ef969f07ecaf5b07c0231d8dc80d30ce1"} err="failed to get container status \"91295946411db4d3a64f25dd9f4f0a0ef969f07ecaf5b07c0231d8dc80d30ce1\": rpc error: code = NotFound desc = could not find container \"91295946411db4d3a64f25dd9f4f0a0ef969f07ecaf5b07c0231d8dc80d30ce1\": container with ID starting with 91295946411db4d3a64f25dd9f4f0a0ef969f07ecaf5b07c0231d8dc80d30ce1 not found: ID does not exist"
Jul 07 11:40:41.348738 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:41.348710    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-180e9350-66f7cb65cc-pdgkr"]
Jul 07 11:40:41.357859 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:41.357829    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-180e9350-66f7cb65cc-pdgkr"]
Jul 07 11:40:42.821385 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:42.821317    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50d956eb-997c-4139-a37c-a2ca924fe391" path="/var/lib/kubelet/pods/50d956eb-997c-4139-a37c-a2ca924fe391/volumes"
Jul 07 11:40:50.864191 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:50.864164    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4v7v8_c8c17c90-3e07-4530-8411-d59cef9a1aff/console-operator/2.log"
Jul 07 11:40:50.865891 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:50.865867    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4v7v8_c8c17c90-3e07-4530-8411-d59cef9a1aff/console-operator/2.log"
Jul 07 11:40:51.050033 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:40:51.049996    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-180e9350/e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm"]
Jul 07 11:41:06.070687 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:41:06.070645    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-180e9350/e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm" podUID="a165b186-347d-4bf9-900a-0c86ddd1fa5a" containerName="main" containerID="cri-o://abbf5c536b75fc68718a1fa358ac66313f518ceee2db712672df839d7ff18414" gracePeriod=45
Jul 07 11:41:06.322627 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:41:06.322564    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-180e9350/e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm"
Jul 07 11:41:06.351017 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:41:06.350983    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a165b186-347d-4bf9-900a-0c86ddd1fa5a-dshm\") pod \"a165b186-347d-4bf9-900a-0c86ddd1fa5a\" (UID: \"a165b186-347d-4bf9-900a-0c86ddd1fa5a\") "
Jul 07 11:41:06.351235 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:41:06.351047    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a165b186-347d-4bf9-900a-0c86ddd1fa5a-tls-certs\") pod \"a165b186-347d-4bf9-900a-0c86ddd1fa5a\" (UID: \"a165b186-347d-4bf9-900a-0c86ddd1fa5a\") "
Jul 07 11:41:06.351235 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:41:06.351109    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a165b186-347d-4bf9-900a-0c86ddd1fa5a-tmp-dir\") pod \"a165b186-347d-4bf9-900a-0c86ddd1fa5a\" (UID: \"a165b186-347d-4bf9-900a-0c86ddd1fa5a\") "
Jul 07 11:41:06.351235 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:41:06.351132    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a165b186-347d-4bf9-900a-0c86ddd1fa5a-model-cache\") pod \"a165b186-347d-4bf9-900a-0c86ddd1fa5a\" (UID: \"a165b186-347d-4bf9-900a-0c86ddd1fa5a\") "
Jul 07 11:41:06.351235 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:41:06.351190    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a165b186-347d-4bf9-900a-0c86ddd1fa5a-home\") pod \"a165b186-347d-4bf9-900a-0c86ddd1fa5a\" (UID: \"a165b186-347d-4bf9-900a-0c86ddd1fa5a\") "
Jul 07 11:41:06.351476 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:41:06.351244    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h9d9k\" (UniqueName: \"kubernetes.io/projected/a165b186-347d-4bf9-900a-0c86ddd1fa5a-kube-api-access-h9d9k\") pod \"a165b186-347d-4bf9-900a-0c86ddd1fa5a\" (UID: \"a165b186-347d-4bf9-900a-0c86ddd1fa5a\") "
Jul 07 11:41:06.351663 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:41:06.351591    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a165b186-347d-4bf9-900a-0c86ddd1fa5a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a165b186-347d-4bf9-900a-0c86ddd1fa5a" (UID: "a165b186-347d-4bf9-900a-0c86ddd1fa5a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:41:06.351938 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:41:06.351794    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a165b186-347d-4bf9-900a-0c86ddd1fa5a-model-cache" (OuterVolumeSpecName: "model-cache") pod "a165b186-347d-4bf9-900a-0c86ddd1fa5a" (UID: "a165b186-347d-4bf9-900a-0c86ddd1fa5a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:41:06.351938 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:41:06.351871    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a165b186-347d-4bf9-900a-0c86ddd1fa5a-home" (OuterVolumeSpecName: "home") pod "a165b186-347d-4bf9-900a-0c86ddd1fa5a" (UID: "a165b186-347d-4bf9-900a-0c86ddd1fa5a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:41:06.354435 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:41:06.354383    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a165b186-347d-4bf9-900a-0c86ddd1fa5a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a165b186-347d-4bf9-900a-0c86ddd1fa5a" (UID: "a165b186-347d-4bf9-900a-0c86ddd1fa5a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 11:41:06.354853 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:41:06.354823    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a165b186-347d-4bf9-900a-0c86ddd1fa5a-kube-api-access-h9d9k" (OuterVolumeSpecName: "kube-api-access-h9d9k") pod "a165b186-347d-4bf9-900a-0c86ddd1fa5a" (UID: "a165b186-347d-4bf9-900a-0c86ddd1fa5a"). InnerVolumeSpecName "kube-api-access-h9d9k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:41:06.354936 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:41:06.354866    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a165b186-347d-4bf9-900a-0c86ddd1fa5a-dshm" (OuterVolumeSpecName: "dshm") pod "a165b186-347d-4bf9-900a-0c86ddd1fa5a" (UID: "a165b186-347d-4bf9-900a-0c86ddd1fa5a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:41:06.447614 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:41:06.447577    2570 generic.go:356] "Generic (PLEG): container finished" podID="a165b186-347d-4bf9-900a-0c86ddd1fa5a" containerID="abbf5c536b75fc68718a1fa358ac66313f518ceee2db712672df839d7ff18414" exitCode=0
Jul 07 11:41:06.448062 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:41:06.447622    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-180e9350/e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm"
Jul 07 11:41:06.448062 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:41:06.447661    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-180e9350/e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm" event={"ID":"a165b186-347d-4bf9-900a-0c86ddd1fa5a","Type":"ContainerDied","Data":"abbf5c536b75fc68718a1fa358ac66313f518ceee2db712672df839d7ff18414"}
Jul 07 11:41:06.448062 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:41:06.447699    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-180e9350/e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm" event={"ID":"a165b186-347d-4bf9-900a-0c86ddd1fa5a","Type":"ContainerDied","Data":"d6c11caf9e80ab2c141de50b8df15b47740be5dd64675e52c4630c60101e48c9"}
Jul 07 11:41:06.448062 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:41:06.447716    2570 scope.go:117] "RemoveContainer" containerID="abbf5c536b75fc68718a1fa358ac66313f518ceee2db712672df839d7ff18414"
Jul 07 11:41:06.452051 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:41:06.452027    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a165b186-347d-4bf9-900a-0c86ddd1fa5a-tls-certs\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:41:06.452051 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:41:06.452051    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a165b186-347d-4bf9-900a-0c86ddd1fa5a-tmp-dir\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:41:06.452196 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:41:06.452064    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a165b186-347d-4bf9-900a-0c86ddd1fa5a-model-cache\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:41:06.452196 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:41:06.452074    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a165b186-347d-4bf9-900a-0c86ddd1fa5a-home\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:41:06.452196 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:41:06.452086    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h9d9k\" (UniqueName: \"kubernetes.io/projected/a165b186-347d-4bf9-900a-0c86ddd1fa5a-kube-api-access-h9d9k\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:41:06.452196 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:41:06.452098    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a165b186-347d-4bf9-900a-0c86ddd1fa5a-dshm\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:41:06.457106 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:41:06.457088    2570 scope.go:117] "RemoveContainer" containerID="abbf5c536b75fc68718a1fa358ac66313f518ceee2db712672df839d7ff18414"
Jul 07 11:41:06.457376 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:41:06.457351    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abbf5c536b75fc68718a1fa358ac66313f518ceee2db712672df839d7ff18414\": container with ID starting with abbf5c536b75fc68718a1fa358ac66313f518ceee2db712672df839d7ff18414 not found: ID does not exist" containerID="abbf5c536b75fc68718a1fa358ac66313f518ceee2db712672df839d7ff18414"
Jul 07 11:41:06.457426 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:41:06.457389    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abbf5c536b75fc68718a1fa358ac66313f518ceee2db712672df839d7ff18414"} err="failed to get container status \"abbf5c536b75fc68718a1fa358ac66313f518ceee2db712672df839d7ff18414\": rpc error: code = NotFound desc = could not find container \"abbf5c536b75fc68718a1fa358ac66313f518ceee2db712672df839d7ff18414\": container with ID starting with abbf5c536b75fc68718a1fa358ac66313f518ceee2db712672df839d7ff18414 not found: ID does not exist"
Jul 07 11:41:06.471744 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:41:06.471719    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-180e9350/e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm"]
Jul 07 11:41:06.478740 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:41:06.478720    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-180e9350/e2e-lifecycle-model-180e9350-kserve-d58f7d79f-b5ksm"]
Jul 07 11:41:06.821386 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:41:06.821352    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a165b186-347d-4bf9-900a-0c86ddd1fa5a" path="/var/lib/kubelet/pods/a165b186-347d-4bf9-900a-0c86ddd1fa5a/volumes"
Jul 07 11:42:34.982568 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:34.982536    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-a-efe248-fc466d848-sf49x"]
Jul 07 11:42:34.983138 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:34.983075    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50d956eb-997c-4139-a37c-a2ca924fe391" containerName="maas-api"
Jul 07 11:42:34.983138 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:34.983095    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="50d956eb-997c-4139-a37c-a2ca924fe391" containerName="maas-api"
Jul 07 11:42:34.983138 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:34.983108    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a165b186-347d-4bf9-900a-0c86ddd1fa5a" containerName="main"
Jul 07 11:42:34.983138 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:34.983118    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="a165b186-347d-4bf9-900a-0c86ddd1fa5a" containerName="main"
Jul 07 11:42:34.983385 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:34.983235    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="a165b186-347d-4bf9-900a-0c86ddd1fa5a" containerName="main"
Jul 07 11:42:34.983385 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:34.983250    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="50d956eb-997c-4139-a37c-a2ca924fe391" containerName="maas-api"
Jul 07 11:42:34.986516 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:34.986494    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-efe248-fc466d848-sf49x"
Jul 07 11:42:34.989485 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:34.989465    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-a-efe248-serving-cert\""
Jul 07 11:42:34.998189 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:34.998167    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-efe248-fc466d848-sf49x"]
Jul 07 11:42:35.136042 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:35.136006    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33693010-00dc-4491-98f1-ab7afc2181d0-maas-api-tls\") pod \"maas-api-e2e-inf-a-efe248-fc466d848-sf49x\" (UID: \"33693010-00dc-4491-98f1-ab7afc2181d0\") " pod="opendatahub/maas-api-e2e-inf-a-efe248-fc466d848-sf49x"
Jul 07 11:42:35.136042 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:35.136045    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33693010-00dc-4491-98f1-ab7afc2181d0-openshift-service-ca\") pod \"maas-api-e2e-inf-a-efe248-fc466d848-sf49x\" (UID: \"33693010-00dc-4491-98f1-ab7afc2181d0\") " pod="opendatahub/maas-api-e2e-inf-a-efe248-fc466d848-sf49x"
Jul 07 11:42:35.136279 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:35.136144    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d58rs\" (UniqueName: \"kubernetes.io/projected/33693010-00dc-4491-98f1-ab7afc2181d0-kube-api-access-d58rs\") pod \"maas-api-e2e-inf-a-efe248-fc466d848-sf49x\" (UID: \"33693010-00dc-4491-98f1-ab7afc2181d0\") " pod="opendatahub/maas-api-e2e-inf-a-efe248-fc466d848-sf49x"
Jul 07 11:42:35.237414 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:35.237296    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d58rs\" (UniqueName: \"kubernetes.io/projected/33693010-00dc-4491-98f1-ab7afc2181d0-kube-api-access-d58rs\") pod \"maas-api-e2e-inf-a-efe248-fc466d848-sf49x\" (UID: \"33693010-00dc-4491-98f1-ab7afc2181d0\") " pod="opendatahub/maas-api-e2e-inf-a-efe248-fc466d848-sf49x"
Jul 07 11:42:35.237414 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:35.237378    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33693010-00dc-4491-98f1-ab7afc2181d0-maas-api-tls\") pod \"maas-api-e2e-inf-a-efe248-fc466d848-sf49x\" (UID: \"33693010-00dc-4491-98f1-ab7afc2181d0\") " pod="opendatahub/maas-api-e2e-inf-a-efe248-fc466d848-sf49x"
Jul 07 11:42:35.237414 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:35.237399    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33693010-00dc-4491-98f1-ab7afc2181d0-openshift-service-ca\") pod \"maas-api-e2e-inf-a-efe248-fc466d848-sf49x\" (UID: \"33693010-00dc-4491-98f1-ab7afc2181d0\") " pod="opendatahub/maas-api-e2e-inf-a-efe248-fc466d848-sf49x"
Jul 07 11:42:35.238001 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:35.237982    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33693010-00dc-4491-98f1-ab7afc2181d0-openshift-service-ca\") pod \"maas-api-e2e-inf-a-efe248-fc466d848-sf49x\" (UID: \"33693010-00dc-4491-98f1-ab7afc2181d0\") " pod="opendatahub/maas-api-e2e-inf-a-efe248-fc466d848-sf49x"
Jul 07 11:42:35.240243 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:35.240220    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33693010-00dc-4491-98f1-ab7afc2181d0-maas-api-tls\") pod \"maas-api-e2e-inf-a-efe248-fc466d848-sf49x\" (UID: \"33693010-00dc-4491-98f1-ab7afc2181d0\") " pod="opendatahub/maas-api-e2e-inf-a-efe248-fc466d848-sf49x"
Jul 07 11:42:35.249987 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:35.249956    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d58rs\" (UniqueName: \"kubernetes.io/projected/33693010-00dc-4491-98f1-ab7afc2181d0-kube-api-access-d58rs\") pod \"maas-api-e2e-inf-a-efe248-fc466d848-sf49x\" (UID: \"33693010-00dc-4491-98f1-ab7afc2181d0\") " pod="opendatahub/maas-api-e2e-inf-a-efe248-fc466d848-sf49x"
Jul 07 11:42:35.297878 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:35.297840    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-efe248-fc466d848-sf49x"
Jul 07 11:42:35.656585 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:42:35.656557    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33693010_00dc_4491_98f1_ab7afc2181d0.slice/crio-9749420e86f40c048a1da1314e7fed491df3197489e0489138b7a4f9fefc6541 WatchSource:0}: Error finding container 9749420e86f40c048a1da1314e7fed491df3197489e0489138b7a4f9fefc6541: Status 404 returned error can't find the container with id 9749420e86f40c048a1da1314e7fed491df3197489e0489138b7a4f9fefc6541
Jul 07 11:42:35.656585 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:35.656559    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-efe248-fc466d848-sf49x"]
Jul 07 11:42:35.658394 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:35.658374    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 11:42:35.876125 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:35.876078    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-efe248-fc466d848-sf49x" event={"ID":"33693010-00dc-4491-98f1-ab7afc2181d0","Type":"ContainerStarted","Data":"9749420e86f40c048a1da1314e7fed491df3197489e0489138b7a4f9fefc6541"}
Jul 07 11:42:36.883103 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:36.883010    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-efe248-fc466d848-sf49x" event={"ID":"33693010-00dc-4491-98f1-ab7afc2181d0","Type":"ContainerStarted","Data":"2939a683a08385725328c80c3173eedff24affc696c6ea6806b55f0b2103be79"}
Jul 07 11:42:36.883579 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:36.883132    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-a-efe248-fc466d848-sf49x"
Jul 07 11:42:36.909179 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:36.909121    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-a-efe248-fc466d848-sf49x" podStartSLOduration=2.059968867 podStartE2EDuration="2.909103945s" totalImagesPullingTime="849.135078ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:42:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:42:36.907864443 +0000 UTC m=+1906.617760832" watchObservedRunningTime="2026-07-07 11:42:36.909103945 +0000 UTC m=+1906.619000332"
Jul 07 11:42:42.893781 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:42.893746    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-a-efe248-fc466d848-sf49x"
Jul 07 11:42:43.013958 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:43.013923    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-b-2507eb-67c9fd9ddd-24tqq"]
Jul 07 11:42:43.017739 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:43.017718    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-2507eb-67c9fd9ddd-24tqq"
Jul 07 11:42:43.022368 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:43.022319    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-b-2507eb-serving-cert\""
Jul 07 11:42:43.052784 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:43.052724    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-2507eb-67c9fd9ddd-24tqq"]
Jul 07 11:42:43.101943 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:43.101908    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7af7e4c-0b25-4f15-8df3-5db9272dc616-maas-api-tls\") pod \"maas-api-e2e-inf-b-2507eb-67c9fd9ddd-24tqq\" (UID: \"b7af7e4c-0b25-4f15-8df3-5db9272dc616\") " pod="opendatahub/maas-api-e2e-inf-b-2507eb-67c9fd9ddd-24tqq"
Jul 07 11:42:43.101943 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:43.101938    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7af7e4c-0b25-4f15-8df3-5db9272dc616-openshift-service-ca\") pod \"maas-api-e2e-inf-b-2507eb-67c9fd9ddd-24tqq\" (UID: \"b7af7e4c-0b25-4f15-8df3-5db9272dc616\") " pod="opendatahub/maas-api-e2e-inf-b-2507eb-67c9fd9ddd-24tqq"
Jul 07 11:42:43.102194 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:43.102039    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nh6l6\" (UniqueName: \"kubernetes.io/projected/b7af7e4c-0b25-4f15-8df3-5db9272dc616-kube-api-access-nh6l6\") pod \"maas-api-e2e-inf-b-2507eb-67c9fd9ddd-24tqq\" (UID: \"b7af7e4c-0b25-4f15-8df3-5db9272dc616\") " pod="opendatahub/maas-api-e2e-inf-b-2507eb-67c9fd9ddd-24tqq"
Jul 07 11:42:43.202517 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:43.202425    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7af7e4c-0b25-4f15-8df3-5db9272dc616-maas-api-tls\") pod \"maas-api-e2e-inf-b-2507eb-67c9fd9ddd-24tqq\" (UID: \"b7af7e4c-0b25-4f15-8df3-5db9272dc616\") " pod="opendatahub/maas-api-e2e-inf-b-2507eb-67c9fd9ddd-24tqq"
Jul 07 11:42:43.202517 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:43.202463    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7af7e4c-0b25-4f15-8df3-5db9272dc616-openshift-service-ca\") pod \"maas-api-e2e-inf-b-2507eb-67c9fd9ddd-24tqq\" (UID: \"b7af7e4c-0b25-4f15-8df3-5db9272dc616\") " pod="opendatahub/maas-api-e2e-inf-b-2507eb-67c9fd9ddd-24tqq"
Jul 07 11:42:43.202718 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:43.202532    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nh6l6\" (UniqueName: \"kubernetes.io/projected/b7af7e4c-0b25-4f15-8df3-5db9272dc616-kube-api-access-nh6l6\") pod \"maas-api-e2e-inf-b-2507eb-67c9fd9ddd-24tqq\" (UID: \"b7af7e4c-0b25-4f15-8df3-5db9272dc616\") " pod="opendatahub/maas-api-e2e-inf-b-2507eb-67c9fd9ddd-24tqq"
Jul 07 11:42:43.202718 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:42:43.202570    2570 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-b-2507eb-serving-cert: secret "maas-api-e2e-inf-b-2507eb-serving-cert" not found
Jul 07 11:42:43.202718 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:42:43.202640    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7af7e4c-0b25-4f15-8df3-5db9272dc616-maas-api-tls podName:b7af7e4c-0b25-4f15-8df3-5db9272dc616 nodeName:}" failed. No retries permitted until 2026-07-07 11:42:43.702622894 +0000 UTC m=+1913.412519265 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b7af7e4c-0b25-4f15-8df3-5db9272dc616-maas-api-tls") pod "maas-api-e2e-inf-b-2507eb-67c9fd9ddd-24tqq" (UID: "b7af7e4c-0b25-4f15-8df3-5db9272dc616") : secret "maas-api-e2e-inf-b-2507eb-serving-cert" not found
Jul 07 11:42:43.203173 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:43.203153    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7af7e4c-0b25-4f15-8df3-5db9272dc616-openshift-service-ca\") pod \"maas-api-e2e-inf-b-2507eb-67c9fd9ddd-24tqq\" (UID: \"b7af7e4c-0b25-4f15-8df3-5db9272dc616\") " pod="opendatahub/maas-api-e2e-inf-b-2507eb-67c9fd9ddd-24tqq"
Jul 07 11:42:43.222876 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:43.222841    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nh6l6\" (UniqueName: \"kubernetes.io/projected/b7af7e4c-0b25-4f15-8df3-5db9272dc616-kube-api-access-nh6l6\") pod \"maas-api-e2e-inf-b-2507eb-67c9fd9ddd-24tqq\" (UID: \"b7af7e4c-0b25-4f15-8df3-5db9272dc616\") " pod="opendatahub/maas-api-e2e-inf-b-2507eb-67c9fd9ddd-24tqq"
Jul 07 11:42:43.707167 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:43.707132    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7af7e4c-0b25-4f15-8df3-5db9272dc616-maas-api-tls\") pod \"maas-api-e2e-inf-b-2507eb-67c9fd9ddd-24tqq\" (UID: \"b7af7e4c-0b25-4f15-8df3-5db9272dc616\") " pod="opendatahub/maas-api-e2e-inf-b-2507eb-67c9fd9ddd-24tqq"
Jul 07 11:42:43.709687 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:43.709666    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7af7e4c-0b25-4f15-8df3-5db9272dc616-maas-api-tls\") pod \"maas-api-e2e-inf-b-2507eb-67c9fd9ddd-24tqq\" (UID: \"b7af7e4c-0b25-4f15-8df3-5db9272dc616\") " pod="opendatahub/maas-api-e2e-inf-b-2507eb-67c9fd9ddd-24tqq"
Jul 07 11:42:43.928190 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:43.928159    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-2507eb-67c9fd9ddd-24tqq"
Jul 07 11:42:44.085201 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:44.085174    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-2507eb-67c9fd9ddd-24tqq"]
Jul 07 11:42:44.087726 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:42:44.087699    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7af7e4c_0b25_4f15_8df3_5db9272dc616.slice/crio-d2991f3e32601c43ddfc0b9b45e7ba0a3929eda477995a6e3b9077d1d65ff9a9 WatchSource:0}: Error finding container d2991f3e32601c43ddfc0b9b45e7ba0a3929eda477995a6e3b9077d1d65ff9a9: Status 404 returned error can't find the container with id d2991f3e32601c43ddfc0b9b45e7ba0a3929eda477995a6e3b9077d1d65ff9a9
Jul 07 11:42:44.922428 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:44.922392    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-2507eb-67c9fd9ddd-24tqq" event={"ID":"b7af7e4c-0b25-4f15-8df3-5db9272dc616","Type":"ContainerStarted","Data":"e3f33ca48c719c28130efb04d306f494de894196796333fc70274c15e6ec4c2c"}
Jul 07 11:42:44.922606 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:44.922454    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-2507eb-67c9fd9ddd-24tqq" event={"ID":"b7af7e4c-0b25-4f15-8df3-5db9272dc616","Type":"ContainerStarted","Data":"d2991f3e32601c43ddfc0b9b45e7ba0a3929eda477995a6e3b9077d1d65ff9a9"}
Jul 07 11:42:44.922606 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:44.922555    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-b-2507eb-67c9fd9ddd-24tqq"
Jul 07 11:42:44.953303 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:44.953242    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-b-2507eb-67c9fd9ddd-24tqq" podStartSLOduration=2.640304471 podStartE2EDuration="2.953226652s" totalImagesPullingTime="312.922181ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:42:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:42:44.95301811 +0000 UTC m=+1914.662914497" watchObservedRunningTime="2026-07-07 11:42:44.953226652 +0000 UTC m=+1914.663123038"
Jul 07 11:42:49.922955 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:49.922911    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-efe248/test-model-efe248-kserve-5fb9b4ddd5-bjkvl"]
Jul 07 11:42:49.927102 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:49.927078    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-efe248/test-model-efe248-kserve-5fb9b4ddd5-bjkvl"
Jul 07 11:42:49.931914 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:49.931889    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-efe248\"/\"kube-root-ca.crt\""
Jul 07 11:42:49.932003 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:49.931971    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-efe248\"/\"openshift-service-ca.crt\""
Jul 07 11:42:49.932003 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:49.931995    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-efe248\"/\"test-model-efe248-kserve-self-signed-certs\""
Jul 07 11:42:49.932101 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:49.931889    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-efe248\"/\"default-dockercfg-hbf8s\""
Jul 07 11:42:49.936217 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:49.936191    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-efe248/test-model-efe248-kserve-5fb9b4ddd5-bjkvl"]
Jul 07 11:42:49.968269 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:49.968231    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ec153bf9-a047-4725-94f6-60752b1601d4-dshm\") pod \"test-model-efe248-kserve-5fb9b4ddd5-bjkvl\" (UID: \"ec153bf9-a047-4725-94f6-60752b1601d4\") " pod="ai-tenant-e2e-inf-a-efe248/test-model-efe248-kserve-5fb9b4ddd5-bjkvl"
Jul 07 11:42:49.968459 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:49.968273    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txsvj\" (UniqueName: \"kubernetes.io/projected/ec153bf9-a047-4725-94f6-60752b1601d4-kube-api-access-txsvj\") pod \"test-model-efe248-kserve-5fb9b4ddd5-bjkvl\" (UID: \"ec153bf9-a047-4725-94f6-60752b1601d4\") " pod="ai-tenant-e2e-inf-a-efe248/test-model-efe248-kserve-5fb9b4ddd5-bjkvl"
Jul 07 11:42:49.968459 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:49.968387    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec153bf9-a047-4725-94f6-60752b1601d4-tmp-dir\") pod \"test-model-efe248-kserve-5fb9b4ddd5-bjkvl\" (UID: \"ec153bf9-a047-4725-94f6-60752b1601d4\") " pod="ai-tenant-e2e-inf-a-efe248/test-model-efe248-kserve-5fb9b4ddd5-bjkvl"
Jul 07 11:42:49.968459 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:49.968414    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec153bf9-a047-4725-94f6-60752b1601d4-tls-certs\") pod \"test-model-efe248-kserve-5fb9b4ddd5-bjkvl\" (UID: \"ec153bf9-a047-4725-94f6-60752b1601d4\") " pod="ai-tenant-e2e-inf-a-efe248/test-model-efe248-kserve-5fb9b4ddd5-bjkvl"
Jul 07 11:42:49.968459 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:49.968432    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ec153bf9-a047-4725-94f6-60752b1601d4-home\") pod \"test-model-efe248-kserve-5fb9b4ddd5-bjkvl\" (UID: \"ec153bf9-a047-4725-94f6-60752b1601d4\") " pod="ai-tenant-e2e-inf-a-efe248/test-model-efe248-kserve-5fb9b4ddd5-bjkvl"
Jul 07 11:42:49.968669 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:49.968547    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ec153bf9-a047-4725-94f6-60752b1601d4-model-cache\") pod \"test-model-efe248-kserve-5fb9b4ddd5-bjkvl\" (UID: \"ec153bf9-a047-4725-94f6-60752b1601d4\") " pod="ai-tenant-e2e-inf-a-efe248/test-model-efe248-kserve-5fb9b4ddd5-bjkvl"
Jul 07 11:42:50.069444 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:50.069398    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ec153bf9-a047-4725-94f6-60752b1601d4-dshm\") pod \"test-model-efe248-kserve-5fb9b4ddd5-bjkvl\" (UID: \"ec153bf9-a047-4725-94f6-60752b1601d4\") " pod="ai-tenant-e2e-inf-a-efe248/test-model-efe248-kserve-5fb9b4ddd5-bjkvl"
Jul 07 11:42:50.069631 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:50.069450    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txsvj\" (UniqueName: \"kubernetes.io/projected/ec153bf9-a047-4725-94f6-60752b1601d4-kube-api-access-txsvj\") pod \"test-model-efe248-kserve-5fb9b4ddd5-bjkvl\" (UID: \"ec153bf9-a047-4725-94f6-60752b1601d4\") " pod="ai-tenant-e2e-inf-a-efe248/test-model-efe248-kserve-5fb9b4ddd5-bjkvl"
Jul 07 11:42:50.069631 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:50.069515    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec153bf9-a047-4725-94f6-60752b1601d4-tmp-dir\") pod \"test-model-efe248-kserve-5fb9b4ddd5-bjkvl\" (UID: \"ec153bf9-a047-4725-94f6-60752b1601d4\") " pod="ai-tenant-e2e-inf-a-efe248/test-model-efe248-kserve-5fb9b4ddd5-bjkvl"
Jul 07 11:42:50.069631 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:50.069544    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec153bf9-a047-4725-94f6-60752b1601d4-tls-certs\") pod \"test-model-efe248-kserve-5fb9b4ddd5-bjkvl\" (UID: \"ec153bf9-a047-4725-94f6-60752b1601d4\") " pod="ai-tenant-e2e-inf-a-efe248/test-model-efe248-kserve-5fb9b4ddd5-bjkvl"
Jul 07 11:42:50.069631 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:50.069570    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ec153bf9-a047-4725-94f6-60752b1601d4-home\") pod \"test-model-efe248-kserve-5fb9b4ddd5-bjkvl\" (UID: \"ec153bf9-a047-4725-94f6-60752b1601d4\") " pod="ai-tenant-e2e-inf-a-efe248/test-model-efe248-kserve-5fb9b4ddd5-bjkvl"
Jul 07 11:42:50.069631 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:50.069622    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ec153bf9-a047-4725-94f6-60752b1601d4-model-cache\") pod \"test-model-efe248-kserve-5fb9b4ddd5-bjkvl\" (UID: \"ec153bf9-a047-4725-94f6-60752b1601d4\") " pod="ai-tenant-e2e-inf-a-efe248/test-model-efe248-kserve-5fb9b4ddd5-bjkvl"
Jul 07 11:42:50.069941 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:50.069913    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec153bf9-a047-4725-94f6-60752b1601d4-tmp-dir\") pod \"test-model-efe248-kserve-5fb9b4ddd5-bjkvl\" (UID: \"ec153bf9-a047-4725-94f6-60752b1601d4\") " pod="ai-tenant-e2e-inf-a-efe248/test-model-efe248-kserve-5fb9b4ddd5-bjkvl"
Jul 07 11:42:50.070047 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:50.070023    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ec153bf9-a047-4725-94f6-60752b1601d4-model-cache\") pod \"test-model-efe248-kserve-5fb9b4ddd5-bjkvl\" (UID: \"ec153bf9-a047-4725-94f6-60752b1601d4\") " pod="ai-tenant-e2e-inf-a-efe248/test-model-efe248-kserve-5fb9b4ddd5-bjkvl"
Jul 07 11:42:50.070210 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:50.070188    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ec153bf9-a047-4725-94f6-60752b1601d4-home\") pod \"test-model-efe248-kserve-5fb9b4ddd5-bjkvl\" (UID: \"ec153bf9-a047-4725-94f6-60752b1601d4\") " pod="ai-tenant-e2e-inf-a-efe248/test-model-efe248-kserve-5fb9b4ddd5-bjkvl"
Jul 07 11:42:50.072162 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:50.072120    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ec153bf9-a047-4725-94f6-60752b1601d4-dshm\") pod \"test-model-efe248-kserve-5fb9b4ddd5-bjkvl\" (UID: \"ec153bf9-a047-4725-94f6-60752b1601d4\") " pod="ai-tenant-e2e-inf-a-efe248/test-model-efe248-kserve-5fb9b4ddd5-bjkvl"
Jul 07 11:42:50.072439 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:50.072418    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec153bf9-a047-4725-94f6-60752b1601d4-tls-certs\") pod \"test-model-efe248-kserve-5fb9b4ddd5-bjkvl\" (UID: \"ec153bf9-a047-4725-94f6-60752b1601d4\") " pod="ai-tenant-e2e-inf-a-efe248/test-model-efe248-kserve-5fb9b4ddd5-bjkvl"
Jul 07 11:42:50.080360 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:50.080318    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-txsvj\" (UniqueName: \"kubernetes.io/projected/ec153bf9-a047-4725-94f6-60752b1601d4-kube-api-access-txsvj\") pod \"test-model-efe248-kserve-5fb9b4ddd5-bjkvl\" (UID: \"ec153bf9-a047-4725-94f6-60752b1601d4\") " pod="ai-tenant-e2e-inf-a-efe248/test-model-efe248-kserve-5fb9b4ddd5-bjkvl"
Jul 07 11:42:50.237767 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:50.237738    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-efe248/test-model-efe248-kserve-5fb9b4ddd5-bjkvl"
Jul 07 11:42:50.382612 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:50.382556    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-efe248/test-model-efe248-kserve-5fb9b4ddd5-bjkvl"]
Jul 07 11:42:50.385187 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:42:50.385159    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podec153bf9_a047_4725_94f6_60752b1601d4.slice/crio-ee06dedcc30625a4e506c4a950c79d11e222ba2767e8a561673c7b386c16af4f WatchSource:0}: Error finding container ee06dedcc30625a4e506c4a950c79d11e222ba2767e8a561673c7b386c16af4f: Status 404 returned error can't find the container with id ee06dedcc30625a4e506c4a950c79d11e222ba2767e8a561673c7b386c16af4f
Jul 07 11:42:50.932239 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:50.932209    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-b-2507eb-67c9fd9ddd-24tqq"
Jul 07 11:42:50.952108 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:50.952073    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-efe248/test-model-efe248-kserve-5fb9b4ddd5-bjkvl" event={"ID":"ec153bf9-a047-4725-94f6-60752b1601d4","Type":"ContainerStarted","Data":"e96f759ed41d4a584a6ac21bb6e3ea7f6289773e82a2c72bea88f1fcfff0aacf"}
Jul 07 11:42:50.952108 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:50.952111    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-efe248/test-model-efe248-kserve-5fb9b4ddd5-bjkvl" event={"ID":"ec153bf9-a047-4725-94f6-60752b1601d4","Type":"ContainerStarted","Data":"ee06dedcc30625a4e506c4a950c79d11e222ba2767e8a561673c7b386c16af4f"}
Jul 07 11:42:50.985092 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:42:50.985040    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-efe248/test-model-efe248-kserve-5fb9b4ddd5-bjkvl" podStartSLOduration=1.985024143 podStartE2EDuration="1.985024143s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:42:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:42:50.983881702 +0000 UTC m=+1920.693778089" watchObservedRunningTime="2026-07-07 11:42:50.985024143 +0000 UTC m=+1920.694920507"
Jul 07 11:43:00.237886 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:00.237846    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-efe248/test-model-efe248-kserve-5fb9b4ddd5-bjkvl"
Jul 07 11:43:00.238283 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:00.237932    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-efe248/test-model-efe248-kserve-5fb9b4ddd5-bjkvl"
Jul 07 11:43:00.250633 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:00.250607    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-efe248/test-model-efe248-kserve-5fb9b4ddd5-bjkvl"
Jul 07 11:43:01.015014 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:01.014977    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-efe248/test-model-efe248-kserve-5fb9b4ddd5-bjkvl"
Jul 07 11:43:40.891657 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:40.891620    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-efe248/test-model-efe248-kserve-5fb9b4ddd5-bjkvl"]
Jul 07 11:43:43.171031 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:43.170940    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-efe248-fc466d848-sf49x"]
Jul 07 11:43:43.171526 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:43.171200    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-a-efe248-fc466d848-sf49x" podUID="33693010-00dc-4491-98f1-ab7afc2181d0" containerName="maas-api" containerID="cri-o://2939a683a08385725328c80c3173eedff24affc696c6ea6806b55f0b2103be79" gracePeriod=30
Jul 07 11:43:43.431450 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:43.431386    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-efe248-fc466d848-sf49x"
Jul 07 11:43:43.537206 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:43.537167    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33693010-00dc-4491-98f1-ab7afc2181d0-maas-api-tls\") pod \"33693010-00dc-4491-98f1-ab7afc2181d0\" (UID: \"33693010-00dc-4491-98f1-ab7afc2181d0\") "
Jul 07 11:43:43.537410 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:43.537220    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d58rs\" (UniqueName: \"kubernetes.io/projected/33693010-00dc-4491-98f1-ab7afc2181d0-kube-api-access-d58rs\") pod \"33693010-00dc-4491-98f1-ab7afc2181d0\" (UID: \"33693010-00dc-4491-98f1-ab7afc2181d0\") "
Jul 07 11:43:43.537410 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:43.537284    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33693010-00dc-4491-98f1-ab7afc2181d0-openshift-service-ca\") pod \"33693010-00dc-4491-98f1-ab7afc2181d0\" (UID: \"33693010-00dc-4491-98f1-ab7afc2181d0\") "
Jul 07 11:43:43.537707 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:43.537674    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33693010-00dc-4491-98f1-ab7afc2181d0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "33693010-00dc-4491-98f1-ab7afc2181d0" (UID: "33693010-00dc-4491-98f1-ab7afc2181d0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 11:43:43.539538 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:43.539509    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33693010-00dc-4491-98f1-ab7afc2181d0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "33693010-00dc-4491-98f1-ab7afc2181d0" (UID: "33693010-00dc-4491-98f1-ab7afc2181d0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 11:43:43.539624 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:43.539547    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33693010-00dc-4491-98f1-ab7afc2181d0-kube-api-access-d58rs" (OuterVolumeSpecName: "kube-api-access-d58rs") pod "33693010-00dc-4491-98f1-ab7afc2181d0" (UID: "33693010-00dc-4491-98f1-ab7afc2181d0"). InnerVolumeSpecName "kube-api-access-d58rs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:43:43.638483 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:43.638449    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33693010-00dc-4491-98f1-ab7afc2181d0-maas-api-tls\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:43:43.638483 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:43.638475    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d58rs\" (UniqueName: \"kubernetes.io/projected/33693010-00dc-4491-98f1-ab7afc2181d0-kube-api-access-d58rs\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:43:43.638483 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:43.638485    2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33693010-00dc-4491-98f1-ab7afc2181d0-openshift-service-ca\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:43:44.212224 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:44.212190    2570 generic.go:356] "Generic (PLEG): container finished" podID="33693010-00dc-4491-98f1-ab7afc2181d0" containerID="2939a683a08385725328c80c3173eedff24affc696c6ea6806b55f0b2103be79" exitCode=0
Jul 07 11:43:44.212646 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:44.212232    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-efe248-fc466d848-sf49x"
Jul 07 11:43:44.212646 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:44.212237    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-efe248-fc466d848-sf49x" event={"ID":"33693010-00dc-4491-98f1-ab7afc2181d0","Type":"ContainerDied","Data":"2939a683a08385725328c80c3173eedff24affc696c6ea6806b55f0b2103be79"}
Jul 07 11:43:44.212646 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:44.212371    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-efe248-fc466d848-sf49x" event={"ID":"33693010-00dc-4491-98f1-ab7afc2181d0","Type":"ContainerDied","Data":"9749420e86f40c048a1da1314e7fed491df3197489e0489138b7a4f9fefc6541"}
Jul 07 11:43:44.212646 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:44.212397    2570 scope.go:117] "RemoveContainer" containerID="2939a683a08385725328c80c3173eedff24affc696c6ea6806b55f0b2103be79"
Jul 07 11:43:44.226427 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:44.226404    2570 scope.go:117] "RemoveContainer" containerID="2939a683a08385725328c80c3173eedff24affc696c6ea6806b55f0b2103be79"
Jul 07 11:43:44.226713 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:43:44.226686    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2939a683a08385725328c80c3173eedff24affc696c6ea6806b55f0b2103be79\": container with ID starting with 2939a683a08385725328c80c3173eedff24affc696c6ea6806b55f0b2103be79 not found: ID does not exist" containerID="2939a683a08385725328c80c3173eedff24affc696c6ea6806b55f0b2103be79"
Jul 07 11:43:44.226783 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:44.226741    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2939a683a08385725328c80c3173eedff24affc696c6ea6806b55f0b2103be79"} err="failed to get container status \"2939a683a08385725328c80c3173eedff24affc696c6ea6806b55f0b2103be79\": rpc error: code = NotFound desc = could not find container \"2939a683a08385725328c80c3173eedff24affc696c6ea6806b55f0b2103be79\": container with ID starting with 2939a683a08385725328c80c3173eedff24affc696c6ea6806b55f0b2103be79 not found: ID does not exist"
Jul 07 11:43:44.241036 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:44.241010    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-efe248-fc466d848-sf49x"]
Jul 07 11:43:44.249259 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:44.249238    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-efe248-fc466d848-sf49x"]
Jul 07 11:43:44.821361 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:44.821303    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33693010-00dc-4491-98f1-ab7afc2181d0" path="/var/lib/kubelet/pods/33693010-00dc-4491-98f1-ab7afc2181d0/volumes"
Jul 07 11:43:55.912132 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:55.912078    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-efe248/test-model-efe248-kserve-5fb9b4ddd5-bjkvl" podUID="ec153bf9-a047-4725-94f6-60752b1601d4" containerName="main" containerID="cri-o://e96f759ed41d4a584a6ac21bb6e3ea7f6289773e82a2c72bea88f1fcfff0aacf" gracePeriod=45
Jul 07 11:43:56.168638 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:56.168576    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-efe248/test-model-efe248-kserve-5fb9b4ddd5-bjkvl"
Jul 07 11:43:56.251566 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:56.251528    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-txsvj\" (UniqueName: \"kubernetes.io/projected/ec153bf9-a047-4725-94f6-60752b1601d4-kube-api-access-txsvj\") pod \"ec153bf9-a047-4725-94f6-60752b1601d4\" (UID: \"ec153bf9-a047-4725-94f6-60752b1601d4\") "
Jul 07 11:43:56.251566 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:56.251568    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec153bf9-a047-4725-94f6-60752b1601d4-tmp-dir\") pod \"ec153bf9-a047-4725-94f6-60752b1601d4\" (UID: \"ec153bf9-a047-4725-94f6-60752b1601d4\") "
Jul 07 11:43:56.251778 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:56.251623    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ec153bf9-a047-4725-94f6-60752b1601d4-model-cache\") pod \"ec153bf9-a047-4725-94f6-60752b1601d4\" (UID: \"ec153bf9-a047-4725-94f6-60752b1601d4\") "
Jul 07 11:43:56.251778 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:56.251646    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec153bf9-a047-4725-94f6-60752b1601d4-tls-certs\") pod \"ec153bf9-a047-4725-94f6-60752b1601d4\" (UID: \"ec153bf9-a047-4725-94f6-60752b1601d4\") "
Jul 07 11:43:56.251778 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:56.251682    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ec153bf9-a047-4725-94f6-60752b1601d4-home\") pod \"ec153bf9-a047-4725-94f6-60752b1601d4\" (UID: \"ec153bf9-a047-4725-94f6-60752b1601d4\") "
Jul 07 11:43:56.251778 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:56.251715    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ec153bf9-a047-4725-94f6-60752b1601d4-dshm\") pod \"ec153bf9-a047-4725-94f6-60752b1601d4\" (UID: \"ec153bf9-a047-4725-94f6-60752b1601d4\") "
Jul 07 11:43:56.251981 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:56.251934    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec153bf9-a047-4725-94f6-60752b1601d4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ec153bf9-a047-4725-94f6-60752b1601d4" (UID: "ec153bf9-a047-4725-94f6-60752b1601d4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:43:56.251981 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:56.251944    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec153bf9-a047-4725-94f6-60752b1601d4-model-cache" (OuterVolumeSpecName: "model-cache") pod "ec153bf9-a047-4725-94f6-60752b1601d4" (UID: "ec153bf9-a047-4725-94f6-60752b1601d4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:43:56.251981 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:56.251963    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec153bf9-a047-4725-94f6-60752b1601d4-home" (OuterVolumeSpecName: "home") pod "ec153bf9-a047-4725-94f6-60752b1601d4" (UID: "ec153bf9-a047-4725-94f6-60752b1601d4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:43:56.254041 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:56.254014    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec153bf9-a047-4725-94f6-60752b1601d4-dshm" (OuterVolumeSpecName: "dshm") pod "ec153bf9-a047-4725-94f6-60752b1601d4" (UID: "ec153bf9-a047-4725-94f6-60752b1601d4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:43:56.254176 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:56.254046    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec153bf9-a047-4725-94f6-60752b1601d4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ec153bf9-a047-4725-94f6-60752b1601d4" (UID: "ec153bf9-a047-4725-94f6-60752b1601d4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 11:43:56.254176 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:56.254050    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec153bf9-a047-4725-94f6-60752b1601d4-kube-api-access-txsvj" (OuterVolumeSpecName: "kube-api-access-txsvj") pod "ec153bf9-a047-4725-94f6-60752b1601d4" (UID: "ec153bf9-a047-4725-94f6-60752b1601d4"). InnerVolumeSpecName "kube-api-access-txsvj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:43:56.271771 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:56.271740    2570 generic.go:356] "Generic (PLEG): container finished" podID="ec153bf9-a047-4725-94f6-60752b1601d4" containerID="e96f759ed41d4a584a6ac21bb6e3ea7f6289773e82a2c72bea88f1fcfff0aacf" exitCode=0
Jul 07 11:43:56.271916 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:56.271794    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-efe248/test-model-efe248-kserve-5fb9b4ddd5-bjkvl"
Jul 07 11:43:56.271916 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:56.271824    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-efe248/test-model-efe248-kserve-5fb9b4ddd5-bjkvl" event={"ID":"ec153bf9-a047-4725-94f6-60752b1601d4","Type":"ContainerDied","Data":"e96f759ed41d4a584a6ac21bb6e3ea7f6289773e82a2c72bea88f1fcfff0aacf"}
Jul 07 11:43:56.271916 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:56.271872    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-efe248/test-model-efe248-kserve-5fb9b4ddd5-bjkvl" event={"ID":"ec153bf9-a047-4725-94f6-60752b1601d4","Type":"ContainerDied","Data":"ee06dedcc30625a4e506c4a950c79d11e222ba2767e8a561673c7b386c16af4f"}
Jul 07 11:43:56.271916 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:56.271890    2570 scope.go:117] "RemoveContainer" containerID="e96f759ed41d4a584a6ac21bb6e3ea7f6289773e82a2c72bea88f1fcfff0aacf"
Jul 07 11:43:56.281977 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:56.281795    2570 scope.go:117] "RemoveContainer" containerID="e96f759ed41d4a584a6ac21bb6e3ea7f6289773e82a2c72bea88f1fcfff0aacf"
Jul 07 11:43:56.283869 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:43:56.283834    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e96f759ed41d4a584a6ac21bb6e3ea7f6289773e82a2c72bea88f1fcfff0aacf\": container with ID starting with e96f759ed41d4a584a6ac21bb6e3ea7f6289773e82a2c72bea88f1fcfff0aacf not found: ID does not exist" containerID="e96f759ed41d4a584a6ac21bb6e3ea7f6289773e82a2c72bea88f1fcfff0aacf"
Jul 07 11:43:56.283978 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:56.283874    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e96f759ed41d4a584a6ac21bb6e3ea7f6289773e82a2c72bea88f1fcfff0aacf"} err="failed to get container status \"e96f759ed41d4a584a6ac21bb6e3ea7f6289773e82a2c72bea88f1fcfff0aacf\": rpc error: code = NotFound desc = could not find container \"e96f759ed41d4a584a6ac21bb6e3ea7f6289773e82a2c72bea88f1fcfff0aacf\": container with ID starting with e96f759ed41d4a584a6ac21bb6e3ea7f6289773e82a2c72bea88f1fcfff0aacf not found: ID does not exist"
Jul 07 11:43:56.297307 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:56.297282    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-efe248/test-model-efe248-kserve-5fb9b4ddd5-bjkvl"]
Jul 07 11:43:56.303150 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:56.303125    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-efe248/test-model-efe248-kserve-5fb9b4ddd5-bjkvl"]
Jul 07 11:43:56.352700 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:56.352666    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ec153bf9-a047-4725-94f6-60752b1601d4-home\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:43:56.352700 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:56.352696    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ec153bf9-a047-4725-94f6-60752b1601d4-dshm\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:43:56.352700 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:56.352706    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-txsvj\" (UniqueName: \"kubernetes.io/projected/ec153bf9-a047-4725-94f6-60752b1601d4-kube-api-access-txsvj\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:43:56.352924 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:56.352716    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec153bf9-a047-4725-94f6-60752b1601d4-tmp-dir\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:43:56.352924 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:56.352726    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ec153bf9-a047-4725-94f6-60752b1601d4-model-cache\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:43:56.352924 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:56.352734    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec153bf9-a047-4725-94f6-60752b1601d4-tls-certs\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:43:56.821630 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:43:56.821593    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec153bf9-a047-4725-94f6-60752b1601d4" path="/var/lib/kubelet/pods/ec153bf9-a047-4725-94f6-60752b1601d4/volumes"
Jul 07 11:44:26.175345 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:26.175293    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-2507eb-67c9fd9ddd-24tqq"]
Jul 07 11:44:26.175845 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:26.175519    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-b-2507eb-67c9fd9ddd-24tqq" podUID="b7af7e4c-0b25-4f15-8df3-5db9272dc616" containerName="maas-api" containerID="cri-o://e3f33ca48c719c28130efb04d306f494de894196796333fc70274c15e6ec4c2c" gracePeriod=30
Jul 07 11:44:26.420694 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:26.420652    2570 generic.go:356] "Generic (PLEG): container finished" podID="b7af7e4c-0b25-4f15-8df3-5db9272dc616" containerID="e3f33ca48c719c28130efb04d306f494de894196796333fc70274c15e6ec4c2c" exitCode=0
Jul 07 11:44:26.420913 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:26.420738    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-2507eb-67c9fd9ddd-24tqq" event={"ID":"b7af7e4c-0b25-4f15-8df3-5db9272dc616","Type":"ContainerDied","Data":"e3f33ca48c719c28130efb04d306f494de894196796333fc70274c15e6ec4c2c"}
Jul 07 11:44:26.420913 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:26.420824    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-2507eb-67c9fd9ddd-24tqq" event={"ID":"b7af7e4c-0b25-4f15-8df3-5db9272dc616","Type":"ContainerDied","Data":"d2991f3e32601c43ddfc0b9b45e7ba0a3929eda477995a6e3b9077d1d65ff9a9"}
Jul 07 11:44:26.420913 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:26.420841    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d2991f3e32601c43ddfc0b9b45e7ba0a3929eda477995a6e3b9077d1d65ff9a9"
Jul 07 11:44:26.430237 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:26.430177    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-2507eb-67c9fd9ddd-24tqq"
Jul 07 11:44:26.521387 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:26.521318    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7af7e4c-0b25-4f15-8df3-5db9272dc616-maas-api-tls\") pod \"b7af7e4c-0b25-4f15-8df3-5db9272dc616\" (UID: \"b7af7e4c-0b25-4f15-8df3-5db9272dc616\") "
Jul 07 11:44:26.521574 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:26.521415    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7af7e4c-0b25-4f15-8df3-5db9272dc616-openshift-service-ca\") pod \"b7af7e4c-0b25-4f15-8df3-5db9272dc616\" (UID: \"b7af7e4c-0b25-4f15-8df3-5db9272dc616\") "
Jul 07 11:44:26.521574 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:26.521511    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nh6l6\" (UniqueName: \"kubernetes.io/projected/b7af7e4c-0b25-4f15-8df3-5db9272dc616-kube-api-access-nh6l6\") pod \"b7af7e4c-0b25-4f15-8df3-5db9272dc616\" (UID: \"b7af7e4c-0b25-4f15-8df3-5db9272dc616\") "
Jul 07 11:44:26.521794 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:26.521771    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7af7e4c-0b25-4f15-8df3-5db9272dc616-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b7af7e4c-0b25-4f15-8df3-5db9272dc616" (UID: "b7af7e4c-0b25-4f15-8df3-5db9272dc616"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 11:44:26.523705 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:26.523682    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7af7e4c-0b25-4f15-8df3-5db9272dc616-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b7af7e4c-0b25-4f15-8df3-5db9272dc616" (UID: "b7af7e4c-0b25-4f15-8df3-5db9272dc616"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 11:44:26.523794 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:26.523739    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7af7e4c-0b25-4f15-8df3-5db9272dc616-kube-api-access-nh6l6" (OuterVolumeSpecName: "kube-api-access-nh6l6") pod "b7af7e4c-0b25-4f15-8df3-5db9272dc616" (UID: "b7af7e4c-0b25-4f15-8df3-5db9272dc616"). InnerVolumeSpecName "kube-api-access-nh6l6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:44:26.622824 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:26.622783    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nh6l6\" (UniqueName: \"kubernetes.io/projected/b7af7e4c-0b25-4f15-8df3-5db9272dc616-kube-api-access-nh6l6\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:44:26.622824 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:26.622818    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b7af7e4c-0b25-4f15-8df3-5db9272dc616-maas-api-tls\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:44:26.622824 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:26.622830    2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7af7e4c-0b25-4f15-8df3-5db9272dc616-openshift-service-ca\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:44:27.426614 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:27.426583    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-2507eb-67c9fd9ddd-24tqq"
Jul 07 11:44:27.445230 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:27.445200    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-2507eb-67c9fd9ddd-24tqq"]
Jul 07 11:44:27.449312 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:27.449288    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-2507eb-67c9fd9ddd-24tqq"]
Jul 07 11:44:28.821189 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:28.821154    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7af7e4c-0b25-4f15-8df3-5db9272dc616" path="/var/lib/kubelet/pods/b7af7e4c-0b25-4f15-8df3-5db9272dc616/volumes"
Jul 07 11:44:36.622889 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:36.622855    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-a-a114ee-5d665c975b-vvqkq"]
Jul 07 11:44:36.623262 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:36.623237    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7af7e4c-0b25-4f15-8df3-5db9272dc616" containerName="maas-api"
Jul 07 11:44:36.623262 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:36.623248    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7af7e4c-0b25-4f15-8df3-5db9272dc616" containerName="maas-api"
Jul 07 11:44:36.623262 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:36.623256    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec153bf9-a047-4725-94f6-60752b1601d4" containerName="main"
Jul 07 11:44:36.623262 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:36.623262    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec153bf9-a047-4725-94f6-60752b1601d4" containerName="main"
Jul 07 11:44:36.623437 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:36.623270    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33693010-00dc-4491-98f1-ab7afc2181d0" containerName="maas-api"
Jul 07 11:44:36.623437 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:36.623277    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="33693010-00dc-4491-98f1-ab7afc2181d0" containerName="maas-api"
Jul 07 11:44:36.623437 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:36.623358    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec153bf9-a047-4725-94f6-60752b1601d4" containerName="main"
Jul 07 11:44:36.623437 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:36.623367    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="33693010-00dc-4491-98f1-ab7afc2181d0" containerName="maas-api"
Jul 07 11:44:36.623437 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:36.623375    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7af7e4c-0b25-4f15-8df3-5db9272dc616" containerName="maas-api"
Jul 07 11:44:36.627642 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:36.627621    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-a114ee-5d665c975b-vvqkq"
Jul 07 11:44:36.632044 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:36.632019    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-a114ee-serving-cert\""
Jul 07 11:44:36.649582 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:36.649554    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-a114ee-5d665c975b-vvqkq"]
Jul 07 11:44:36.704752 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:36.704708    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfjhs\" (UniqueName: \"kubernetes.io/projected/336618a1-32e2-4b9b-ac47-9e123d4edda0-kube-api-access-rfjhs\") pod \"maas-api-e2e-api-a-a114ee-5d665c975b-vvqkq\" (UID: \"336618a1-32e2-4b9b-ac47-9e123d4edda0\") " pod="opendatahub/maas-api-e2e-api-a-a114ee-5d665c975b-vvqkq"
Jul 07 11:44:36.704936 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:36.704775    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/336618a1-32e2-4b9b-ac47-9e123d4edda0-openshift-service-ca\") pod \"maas-api-e2e-api-a-a114ee-5d665c975b-vvqkq\" (UID: \"336618a1-32e2-4b9b-ac47-9e123d4edda0\") " pod="opendatahub/maas-api-e2e-api-a-a114ee-5d665c975b-vvqkq"
Jul 07 11:44:36.704936 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:36.704877    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/336618a1-32e2-4b9b-ac47-9e123d4edda0-maas-api-tls\") pod \"maas-api-e2e-api-a-a114ee-5d665c975b-vvqkq\" (UID: \"336618a1-32e2-4b9b-ac47-9e123d4edda0\") " pod="opendatahub/maas-api-e2e-api-a-a114ee-5d665c975b-vvqkq"
Jul 07 11:44:36.805984 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:36.805932    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfjhs\" (UniqueName: \"kubernetes.io/projected/336618a1-32e2-4b9b-ac47-9e123d4edda0-kube-api-access-rfjhs\") pod \"maas-api-e2e-api-a-a114ee-5d665c975b-vvqkq\" (UID: \"336618a1-32e2-4b9b-ac47-9e123d4edda0\") " pod="opendatahub/maas-api-e2e-api-a-a114ee-5d665c975b-vvqkq"
Jul 07 11:44:36.805984 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:36.805999    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/336618a1-32e2-4b9b-ac47-9e123d4edda0-openshift-service-ca\") pod \"maas-api-e2e-api-a-a114ee-5d665c975b-vvqkq\" (UID: \"336618a1-32e2-4b9b-ac47-9e123d4edda0\") " pod="opendatahub/maas-api-e2e-api-a-a114ee-5d665c975b-vvqkq"
Jul 07 11:44:36.806261 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:36.806045    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/336618a1-32e2-4b9b-ac47-9e123d4edda0-maas-api-tls\") pod \"maas-api-e2e-api-a-a114ee-5d665c975b-vvqkq\" (UID: \"336618a1-32e2-4b9b-ac47-9e123d4edda0\") " pod="opendatahub/maas-api-e2e-api-a-a114ee-5d665c975b-vvqkq"
Jul 07 11:44:36.806261 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:44:36.806148    2570 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-a-a114ee-serving-cert: secret "maas-api-e2e-api-a-a114ee-serving-cert" not found
Jul 07 11:44:36.806261 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:44:36.806206    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/336618a1-32e2-4b9b-ac47-9e123d4edda0-maas-api-tls podName:336618a1-32e2-4b9b-ac47-9e123d4edda0 nodeName:}" failed. No retries permitted until 2026-07-07 11:44:37.306188223 +0000 UTC m=+2027.016084603 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/336618a1-32e2-4b9b-ac47-9e123d4edda0-maas-api-tls") pod "maas-api-e2e-api-a-a114ee-5d665c975b-vvqkq" (UID: "336618a1-32e2-4b9b-ac47-9e123d4edda0") : secret "maas-api-e2e-api-a-a114ee-serving-cert" not found
Jul 07 11:44:36.806677 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:36.806655    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/336618a1-32e2-4b9b-ac47-9e123d4edda0-openshift-service-ca\") pod \"maas-api-e2e-api-a-a114ee-5d665c975b-vvqkq\" (UID: \"336618a1-32e2-4b9b-ac47-9e123d4edda0\") " pod="opendatahub/maas-api-e2e-api-a-a114ee-5d665c975b-vvqkq"
Jul 07 11:44:36.827142 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:36.827105    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfjhs\" (UniqueName: \"kubernetes.io/projected/336618a1-32e2-4b9b-ac47-9e123d4edda0-kube-api-access-rfjhs\") pod \"maas-api-e2e-api-a-a114ee-5d665c975b-vvqkq\" (UID: \"336618a1-32e2-4b9b-ac47-9e123d4edda0\") " pod="opendatahub/maas-api-e2e-api-a-a114ee-5d665c975b-vvqkq"
Jul 07 11:44:37.310225 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:37.310180    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/336618a1-32e2-4b9b-ac47-9e123d4edda0-maas-api-tls\") pod \"maas-api-e2e-api-a-a114ee-5d665c975b-vvqkq\" (UID: \"336618a1-32e2-4b9b-ac47-9e123d4edda0\") " pod="opendatahub/maas-api-e2e-api-a-a114ee-5d665c975b-vvqkq"
Jul 07 11:44:37.313013 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:37.312975    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/336618a1-32e2-4b9b-ac47-9e123d4edda0-maas-api-tls\") pod \"maas-api-e2e-api-a-a114ee-5d665c975b-vvqkq\" (UID: \"336618a1-32e2-4b9b-ac47-9e123d4edda0\") " pod="opendatahub/maas-api-e2e-api-a-a114ee-5d665c975b-vvqkq"
Jul 07 11:44:37.537462 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:37.537416    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-a114ee-5d665c975b-vvqkq"
Jul 07 11:44:37.701221 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:37.701191    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-a114ee-5d665c975b-vvqkq"]
Jul 07 11:44:37.701935 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:44:37.701910    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod336618a1_32e2_4b9b_ac47_9e123d4edda0.slice/crio-bdbca5209bbc1e2eb4715b66bcbc0078e4cfc574b854cc332bf114c628920823 WatchSource:0}: Error finding container bdbca5209bbc1e2eb4715b66bcbc0078e4cfc574b854cc332bf114c628920823: Status 404 returned error can't find the container with id bdbca5209bbc1e2eb4715b66bcbc0078e4cfc574b854cc332bf114c628920823
Jul 07 11:44:38.492812 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:38.492780    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-a114ee-5d665c975b-vvqkq" event={"ID":"336618a1-32e2-4b9b-ac47-9e123d4edda0","Type":"ContainerStarted","Data":"aadf67b98b5b46c3fb04dcbe2caa1da92c7399304d5398425f60e0d35fa1015c"}
Jul 07 11:44:38.492987 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:38.492835    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-a114ee-5d665c975b-vvqkq" event={"ID":"336618a1-32e2-4b9b-ac47-9e123d4edda0","Type":"ContainerStarted","Data":"bdbca5209bbc1e2eb4715b66bcbc0078e4cfc574b854cc332bf114c628920823"}
Jul 07 11:44:38.492987 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:38.492870    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-a-a114ee-5d665c975b-vvqkq"
Jul 07 11:44:38.537916 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:38.537820    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-a-a114ee-5d665c975b-vvqkq" podStartSLOduration=2.1310971260000002 podStartE2EDuration="2.537803188s" totalImagesPullingTime="406.706062ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:44:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:44:38.53438836 +0000 UTC m=+2028.244284748" watchObservedRunningTime="2026-07-07 11:44:38.537803188 +0000 UTC m=+2028.247699574"
Jul 07 11:44:44.503499 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:44.503455    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-a-a114ee-5d665c975b-vvqkq"
Jul 07 11:44:44.732390 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:44.732350    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-b-cd7fa7-7c5f897c5b-fwzhl"]
Jul 07 11:44:44.736116 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:44.736093    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-cd7fa7-7c5f897c5b-fwzhl"
Jul 07 11:44:44.741622 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:44.741601    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-b-cd7fa7-serving-cert\""
Jul 07 11:44:44.748954 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:44.748934    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-cd7fa7-7c5f897c5b-fwzhl"]
Jul 07 11:44:44.777424 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:44.777317    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51389f51-4770-40ac-a3dd-f71667857a4b-openshift-service-ca\") pod \"maas-api-e2e-api-b-cd7fa7-7c5f897c5b-fwzhl\" (UID: \"51389f51-4770-40ac-a3dd-f71667857a4b\") " pod="opendatahub/maas-api-e2e-api-b-cd7fa7-7c5f897c5b-fwzhl"
Jul 07 11:44:44.777572 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:44.777427    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51389f51-4770-40ac-a3dd-f71667857a4b-maas-api-tls\") pod \"maas-api-e2e-api-b-cd7fa7-7c5f897c5b-fwzhl\" (UID: \"51389f51-4770-40ac-a3dd-f71667857a4b\") " pod="opendatahub/maas-api-e2e-api-b-cd7fa7-7c5f897c5b-fwzhl"
Jul 07 11:44:44.777572 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:44.777477    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hgfr\" (UniqueName: \"kubernetes.io/projected/51389f51-4770-40ac-a3dd-f71667857a4b-kube-api-access-9hgfr\") pod \"maas-api-e2e-api-b-cd7fa7-7c5f897c5b-fwzhl\" (UID: \"51389f51-4770-40ac-a3dd-f71667857a4b\") " pod="opendatahub/maas-api-e2e-api-b-cd7fa7-7c5f897c5b-fwzhl"
Jul 07 11:44:44.878561 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:44.878523    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51389f51-4770-40ac-a3dd-f71667857a4b-openshift-service-ca\") pod \"maas-api-e2e-api-b-cd7fa7-7c5f897c5b-fwzhl\" (UID: \"51389f51-4770-40ac-a3dd-f71667857a4b\") " pod="opendatahub/maas-api-e2e-api-b-cd7fa7-7c5f897c5b-fwzhl"
Jul 07 11:44:44.878759 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:44.878605    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51389f51-4770-40ac-a3dd-f71667857a4b-maas-api-tls\") pod \"maas-api-e2e-api-b-cd7fa7-7c5f897c5b-fwzhl\" (UID: \"51389f51-4770-40ac-a3dd-f71667857a4b\") " pod="opendatahub/maas-api-e2e-api-b-cd7fa7-7c5f897c5b-fwzhl"
Jul 07 11:44:44.878759 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:44.878661    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hgfr\" (UniqueName: \"kubernetes.io/projected/51389f51-4770-40ac-a3dd-f71667857a4b-kube-api-access-9hgfr\") pod \"maas-api-e2e-api-b-cd7fa7-7c5f897c5b-fwzhl\" (UID: \"51389f51-4770-40ac-a3dd-f71667857a4b\") " pod="opendatahub/maas-api-e2e-api-b-cd7fa7-7c5f897c5b-fwzhl"
Jul 07 11:44:44.878883 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:44:44.878757    2570 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-b-cd7fa7-serving-cert: secret "maas-api-e2e-api-b-cd7fa7-serving-cert" not found
Jul 07 11:44:44.878883 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:44:44.878827    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51389f51-4770-40ac-a3dd-f71667857a4b-maas-api-tls podName:51389f51-4770-40ac-a3dd-f71667857a4b nodeName:}" failed. No retries permitted until 2026-07-07 11:44:45.378811472 +0000 UTC m=+2035.088707841 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/51389f51-4770-40ac-a3dd-f71667857a4b-maas-api-tls") pod "maas-api-e2e-api-b-cd7fa7-7c5f897c5b-fwzhl" (UID: "51389f51-4770-40ac-a3dd-f71667857a4b") : secret "maas-api-e2e-api-b-cd7fa7-serving-cert" not found
Jul 07 11:44:44.879170 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:44.879150    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51389f51-4770-40ac-a3dd-f71667857a4b-openshift-service-ca\") pod \"maas-api-e2e-api-b-cd7fa7-7c5f897c5b-fwzhl\" (UID: \"51389f51-4770-40ac-a3dd-f71667857a4b\") " pod="opendatahub/maas-api-e2e-api-b-cd7fa7-7c5f897c5b-fwzhl"
Jul 07 11:44:44.913836 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:44.913802    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hgfr\" (UniqueName: \"kubernetes.io/projected/51389f51-4770-40ac-a3dd-f71667857a4b-kube-api-access-9hgfr\") pod \"maas-api-e2e-api-b-cd7fa7-7c5f897c5b-fwzhl\" (UID: \"51389f51-4770-40ac-a3dd-f71667857a4b\") " pod="opendatahub/maas-api-e2e-api-b-cd7fa7-7c5f897c5b-fwzhl"
Jul 07 11:44:45.381890 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:45.381855    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51389f51-4770-40ac-a3dd-f71667857a4b-maas-api-tls\") pod \"maas-api-e2e-api-b-cd7fa7-7c5f897c5b-fwzhl\" (UID: \"51389f51-4770-40ac-a3dd-f71667857a4b\") " pod="opendatahub/maas-api-e2e-api-b-cd7fa7-7c5f897c5b-fwzhl"
Jul 07 11:44:45.384434 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:45.384411    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51389f51-4770-40ac-a3dd-f71667857a4b-maas-api-tls\") pod \"maas-api-e2e-api-b-cd7fa7-7c5f897c5b-fwzhl\" (UID: \"51389f51-4770-40ac-a3dd-f71667857a4b\") " pod="opendatahub/maas-api-e2e-api-b-cd7fa7-7c5f897c5b-fwzhl"
Jul 07 11:44:45.646445 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:45.646356    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-cd7fa7-7c5f897c5b-fwzhl"
Jul 07 11:44:45.819161 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:44:45.819131    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51389f51_4770_40ac_a3dd_f71667857a4b.slice/crio-8bb7845bb0d7b1f9e36029a3e8d82729230e4098e9a3c06c989b576e8e9a6dc9 WatchSource:0}: Error finding container 8bb7845bb0d7b1f9e36029a3e8d82729230e4098e9a3c06c989b576e8e9a6dc9: Status 404 returned error can't find the container with id 8bb7845bb0d7b1f9e36029a3e8d82729230e4098e9a3c06c989b576e8e9a6dc9
Jul 07 11:44:45.823641 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:45.823616    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-cd7fa7-7c5f897c5b-fwzhl"]
Jul 07 11:44:46.533734 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:46.533699    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-cd7fa7-7c5f897c5b-fwzhl" event={"ID":"51389f51-4770-40ac-a3dd-f71667857a4b","Type":"ContainerStarted","Data":"71ee7eca8ac377ca2ce70e508024b50a1b43212ac80f7981d86c65c5dbb7bf4d"}
Jul 07 11:44:46.533900 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:46.533759    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-cd7fa7-7c5f897c5b-fwzhl" event={"ID":"51389f51-4770-40ac-a3dd-f71667857a4b","Type":"ContainerStarted","Data":"8bb7845bb0d7b1f9e36029a3e8d82729230e4098e9a3c06c989b576e8e9a6dc9"}
Jul 07 11:44:46.533964 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:46.533907    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-b-cd7fa7-7c5f897c5b-fwzhl"
Jul 07 11:44:46.552199 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:46.552136    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-b-cd7fa7-7c5f897c5b-fwzhl" podStartSLOduration=2.168114116 podStartE2EDuration="2.552118152s" totalImagesPullingTime="384.004036ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:44:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:44:46.552042431 +0000 UTC m=+2036.261938819" watchObservedRunningTime="2026-07-07 11:44:46.552118152 +0000 UTC m=+2036.262014541"
Jul 07 11:44:52.543713 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:52.543682    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-b-cd7fa7-7c5f897c5b-fwzhl"
Jul 07 11:44:57.753968 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:57.753932    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-a114ee-5d665c975b-vvqkq"]
Jul 07 11:44:57.754851 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:57.754819    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-a-a114ee-5d665c975b-vvqkq" podUID="336618a1-32e2-4b9b-ac47-9e123d4edda0" containerName="maas-api" containerID="cri-o://aadf67b98b5b46c3fb04dcbe2caa1da92c7399304d5398425f60e0d35fa1015c" gracePeriod=30
Jul 07 11:44:58.018250 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:58.018176    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-a114ee-5d665c975b-vvqkq"
Jul 07 11:44:58.100051 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:58.100018    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rfjhs\" (UniqueName: \"kubernetes.io/projected/336618a1-32e2-4b9b-ac47-9e123d4edda0-kube-api-access-rfjhs\") pod \"336618a1-32e2-4b9b-ac47-9e123d4edda0\" (UID: \"336618a1-32e2-4b9b-ac47-9e123d4edda0\") "
Jul 07 11:44:58.100215 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:58.100059    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/336618a1-32e2-4b9b-ac47-9e123d4edda0-maas-api-tls\") pod \"336618a1-32e2-4b9b-ac47-9e123d4edda0\" (UID: \"336618a1-32e2-4b9b-ac47-9e123d4edda0\") "
Jul 07 11:44:58.100215 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:58.100189    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/336618a1-32e2-4b9b-ac47-9e123d4edda0-openshift-service-ca\") pod \"336618a1-32e2-4b9b-ac47-9e123d4edda0\" (UID: \"336618a1-32e2-4b9b-ac47-9e123d4edda0\") "
Jul 07 11:44:58.100629 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:58.100593    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/336618a1-32e2-4b9b-ac47-9e123d4edda0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "336618a1-32e2-4b9b-ac47-9e123d4edda0" (UID: "336618a1-32e2-4b9b-ac47-9e123d4edda0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 11:44:58.102259 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:58.102221    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/336618a1-32e2-4b9b-ac47-9e123d4edda0-kube-api-access-rfjhs" (OuterVolumeSpecName: "kube-api-access-rfjhs") pod "336618a1-32e2-4b9b-ac47-9e123d4edda0" (UID: "336618a1-32e2-4b9b-ac47-9e123d4edda0"). InnerVolumeSpecName "kube-api-access-rfjhs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:44:58.102396 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:58.102220    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/336618a1-32e2-4b9b-ac47-9e123d4edda0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "336618a1-32e2-4b9b-ac47-9e123d4edda0" (UID: "336618a1-32e2-4b9b-ac47-9e123d4edda0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 11:44:58.201217 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:58.201179    2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/336618a1-32e2-4b9b-ac47-9e123d4edda0-openshift-service-ca\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:44:58.201217 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:58.201204    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rfjhs\" (UniqueName: \"kubernetes.io/projected/336618a1-32e2-4b9b-ac47-9e123d4edda0-kube-api-access-rfjhs\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:44:58.201217 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:58.201214    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/336618a1-32e2-4b9b-ac47-9e123d4edda0-maas-api-tls\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:44:58.594391 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:58.594354    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-a114ee-5d665c975b-vvqkq" event={"ID":"336618a1-32e2-4b9b-ac47-9e123d4edda0","Type":"ContainerDied","Data":"aadf67b98b5b46c3fb04dcbe2caa1da92c7399304d5398425f60e0d35fa1015c"}
Jul 07 11:44:58.594391 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:58.594384    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-a114ee-5d665c975b-vvqkq"
Jul 07 11:44:58.594653 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:58.594430    2570 scope.go:117] "RemoveContainer" containerID="aadf67b98b5b46c3fb04dcbe2caa1da92c7399304d5398425f60e0d35fa1015c"
Jul 07 11:44:58.594653 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:58.594321    2570 generic.go:356] "Generic (PLEG): container finished" podID="336618a1-32e2-4b9b-ac47-9e123d4edda0" containerID="aadf67b98b5b46c3fb04dcbe2caa1da92c7399304d5398425f60e0d35fa1015c" exitCode=0
Jul 07 11:44:58.594653 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:58.594526    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-a114ee-5d665c975b-vvqkq" event={"ID":"336618a1-32e2-4b9b-ac47-9e123d4edda0","Type":"ContainerDied","Data":"bdbca5209bbc1e2eb4715b66bcbc0078e4cfc574b854cc332bf114c628920823"}
Jul 07 11:44:58.605259 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:58.605230    2570 scope.go:117] "RemoveContainer" containerID="aadf67b98b5b46c3fb04dcbe2caa1da92c7399304d5398425f60e0d35fa1015c"
Jul 07 11:44:58.605593 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:44:58.605568    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aadf67b98b5b46c3fb04dcbe2caa1da92c7399304d5398425f60e0d35fa1015c\": container with ID starting with aadf67b98b5b46c3fb04dcbe2caa1da92c7399304d5398425f60e0d35fa1015c not found: ID does not exist" containerID="aadf67b98b5b46c3fb04dcbe2caa1da92c7399304d5398425f60e0d35fa1015c"
Jul 07 11:44:58.605683 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:58.605603    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aadf67b98b5b46c3fb04dcbe2caa1da92c7399304d5398425f60e0d35fa1015c"} err="failed to get container status \"aadf67b98b5b46c3fb04dcbe2caa1da92c7399304d5398425f60e0d35fa1015c\": rpc error: code = NotFound desc = could not find container \"aadf67b98b5b46c3fb04dcbe2caa1da92c7399304d5398425f60e0d35fa1015c\": container with ID starting with aadf67b98b5b46c3fb04dcbe2caa1da92c7399304d5398425f60e0d35fa1015c not found: ID does not exist"
Jul 07 11:44:58.620777 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:58.620403    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-a114ee-5d665c975b-vvqkq"]
Jul 07 11:44:58.626419 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:58.626389    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-a-a114ee-5d665c975b-vvqkq"]
Jul 07 11:44:58.820588 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:44:58.820555    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="336618a1-32e2-4b9b-ac47-9e123d4edda0" path="/var/lib/kubelet/pods/336618a1-32e2-4b9b-ac47-9e123d4edda0/volumes"
Jul 07 11:45:04.958589 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:45:04.958512    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-cd7fa7-7c5f897c5b-fwzhl"]
Jul 07 11:45:04.959098 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:45:04.958772    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-b-cd7fa7-7c5f897c5b-fwzhl" podUID="51389f51-4770-40ac-a3dd-f71667857a4b" containerName="maas-api" containerID="cri-o://71ee7eca8ac377ca2ce70e508024b50a1b43212ac80f7981d86c65c5dbb7bf4d" gracePeriod=30
Jul 07 11:45:05.216754 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:45:05.216681    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-cd7fa7-7c5f897c5b-fwzhl"
Jul 07 11:45:05.365606 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:45:05.365572    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51389f51-4770-40ac-a3dd-f71667857a4b-openshift-service-ca\") pod \"51389f51-4770-40ac-a3dd-f71667857a4b\" (UID: \"51389f51-4770-40ac-a3dd-f71667857a4b\") "
Jul 07 11:45:05.365824 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:45:05.365661    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9hgfr\" (UniqueName: \"kubernetes.io/projected/51389f51-4770-40ac-a3dd-f71667857a4b-kube-api-access-9hgfr\") pod \"51389f51-4770-40ac-a3dd-f71667857a4b\" (UID: \"51389f51-4770-40ac-a3dd-f71667857a4b\") "
Jul 07 11:45:05.365824 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:45:05.365695    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51389f51-4770-40ac-a3dd-f71667857a4b-maas-api-tls\") pod \"51389f51-4770-40ac-a3dd-f71667857a4b\" (UID: \"51389f51-4770-40ac-a3dd-f71667857a4b\") "
Jul 07 11:45:05.365981 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:45:05.365954    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51389f51-4770-40ac-a3dd-f71667857a4b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "51389f51-4770-40ac-a3dd-f71667857a4b" (UID: "51389f51-4770-40ac-a3dd-f71667857a4b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 11:45:05.367918 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:45:05.367893    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51389f51-4770-40ac-a3dd-f71667857a4b-kube-api-access-9hgfr" (OuterVolumeSpecName: "kube-api-access-9hgfr") pod "51389f51-4770-40ac-a3dd-f71667857a4b" (UID: "51389f51-4770-40ac-a3dd-f71667857a4b"). InnerVolumeSpecName "kube-api-access-9hgfr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:45:05.368012 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:45:05.367934    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51389f51-4770-40ac-a3dd-f71667857a4b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "51389f51-4770-40ac-a3dd-f71667857a4b" (UID: "51389f51-4770-40ac-a3dd-f71667857a4b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 11:45:05.466673 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:45:05.466630    2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51389f51-4770-40ac-a3dd-f71667857a4b-openshift-service-ca\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:45:05.466673 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:45:05.466666    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9hgfr\" (UniqueName: \"kubernetes.io/projected/51389f51-4770-40ac-a3dd-f71667857a4b-kube-api-access-9hgfr\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:45:05.466941 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:45:05.466683    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51389f51-4770-40ac-a3dd-f71667857a4b-maas-api-tls\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:45:05.630436 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:45:05.630395    2570 generic.go:356] "Generic (PLEG): container finished" podID="51389f51-4770-40ac-a3dd-f71667857a4b" containerID="71ee7eca8ac377ca2ce70e508024b50a1b43212ac80f7981d86c65c5dbb7bf4d" exitCode=0
Jul 07 11:45:05.630640 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:45:05.630447    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-cd7fa7-7c5f897c5b-fwzhl"
Jul 07 11:45:05.630640 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:45:05.630491    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-cd7fa7-7c5f897c5b-fwzhl" event={"ID":"51389f51-4770-40ac-a3dd-f71667857a4b","Type":"ContainerDied","Data":"71ee7eca8ac377ca2ce70e508024b50a1b43212ac80f7981d86c65c5dbb7bf4d"}
Jul 07 11:45:05.630640 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:45:05.630560    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-cd7fa7-7c5f897c5b-fwzhl" event={"ID":"51389f51-4770-40ac-a3dd-f71667857a4b","Type":"ContainerDied","Data":"8bb7845bb0d7b1f9e36029a3e8d82729230e4098e9a3c06c989b576e8e9a6dc9"}
Jul 07 11:45:05.630640 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:45:05.630594    2570 scope.go:117] "RemoveContainer" containerID="71ee7eca8ac377ca2ce70e508024b50a1b43212ac80f7981d86c65c5dbb7bf4d"
Jul 07 11:45:05.640166 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:45:05.640145    2570 scope.go:117] "RemoveContainer" containerID="71ee7eca8ac377ca2ce70e508024b50a1b43212ac80f7981d86c65c5dbb7bf4d"
Jul 07 11:45:05.640448 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:45:05.640429    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71ee7eca8ac377ca2ce70e508024b50a1b43212ac80f7981d86c65c5dbb7bf4d\": container with ID starting with 71ee7eca8ac377ca2ce70e508024b50a1b43212ac80f7981d86c65c5dbb7bf4d not found: ID does not exist" containerID="71ee7eca8ac377ca2ce70e508024b50a1b43212ac80f7981d86c65c5dbb7bf4d"
Jul 07 11:45:05.640502 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:45:05.640458    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71ee7eca8ac377ca2ce70e508024b50a1b43212ac80f7981d86c65c5dbb7bf4d"} err="failed to get container status \"71ee7eca8ac377ca2ce70e508024b50a1b43212ac80f7981d86c65c5dbb7bf4d\": rpc error: code = NotFound desc = could not find container \"71ee7eca8ac377ca2ce70e508024b50a1b43212ac80f7981d86c65c5dbb7bf4d\": container with ID starting with 71ee7eca8ac377ca2ce70e508024b50a1b43212ac80f7981d86c65c5dbb7bf4d not found: ID does not exist"
Jul 07 11:45:05.669041 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:45:05.669007    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-cd7fa7-7c5f897c5b-fwzhl"]
Jul 07 11:45:05.677614 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:45:05.677583    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-b-cd7fa7-7c5f897c5b-fwzhl"]
Jul 07 11:45:06.821561 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:45:06.821525    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51389f51-4770-40ac-a3dd-f71667857a4b" path="/var/lib/kubelet/pods/51389f51-4770-40ac-a3dd-f71667857a4b/volumes"
Jul 07 11:45:14.729790 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:45:14.729703    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-a-b5d0aa-65cbc9b5f5-g4bkh"]
Jul 07 11:45:14.730146 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:45:14.730078    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51389f51-4770-40ac-a3dd-f71667857a4b" containerName="maas-api"
Jul 07 11:45:14.730146 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:45:14.730091    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="51389f51-4770-40ac-a3dd-f71667857a4b" containerName="maas-api"
Jul 07 11:45:14.730146 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:45:14.730113    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="336618a1-32e2-4b9b-ac47-9e123d4edda0" containerName="maas-api"
Jul 07 11:45:14.730146 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:45:14.730119    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="336618a1-32e2-4b9b-ac47-9e123d4edda0" containerName="maas-api"
Jul 07 11:45:14.730276 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:45:14.730178    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="336618a1-32e2-4b9b-ac47-9e123d4edda0" containerName="maas-api"
Jul 07 11:45:14.730276 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:45:14.730185    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="51389f51-4770-40ac-a3dd-f71667857a4b" containerName="maas-api"
Jul 07 11:45:14.734345 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:45:14.734309    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-b5d0aa-65cbc9b5f5-g4bkh"
Jul 07 11:45:14.737656 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:45:14.737636    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-a-b5d0aa-serving-cert\""
Jul 07 11:45:14.747457 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:45:14.747427    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-b5d0aa-65cbc9b5f5-g4bkh"]
Jul 07 11:45:14.858211 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:45:14.858174    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e-maas-api-tls\") pod \"maas-api-e2e-shared-a-b5d0aa-65cbc9b5f5-g4bkh\" (UID: \"2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e\") " pod="opendatahub/maas-api-e2e-shared-a-b5d0aa-65cbc9b5f5-g4bkh"
Jul 07 11:45:14.858425 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:45:14.858231    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcn86\" (UniqueName: \"kubernetes.io/projected/2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e-kube-api-access-xcn86\") pod \"maas-api-e2e-shared-a-b5d0aa-65cbc9b5f5-g4bkh\" (UID: \"2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e\") " pod="opendatahub/maas-api-e2e-shared-a-b5d0aa-65cbc9b5f5-g4bkh"
Jul 07 11:45:14.858425 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:45:14.858366    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e-openshift-service-ca\") pod \"maas-api-e2e-shared-a-b5d0aa-65cbc9b5f5-g4bkh\" (UID: \"2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e\") " pod="opendatahub/maas-api-e2e-shared-a-b5d0aa-65cbc9b5f5-g4bkh"
Jul 07 11:45:14.959196 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:45:14.959160    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e-openshift-service-ca\") pod \"maas-api-e2e-shared-a-b5d0aa-65cbc9b5f5-g4bkh\" (UID: \"2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e\") " pod="opendatahub/maas-api-e2e-shared-a-b5d0aa-65cbc9b5f5-g4bkh"
Jul 07 11:45:14.959196 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:45:14.959198    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e-maas-api-tls\") pod \"maas-api-e2e-shared-a-b5d0aa-65cbc9b5f5-g4bkh\" (UID: \"2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e\") " pod="opendatahub/maas-api-e2e-shared-a-b5d0aa-65cbc9b5f5-g4bkh"
Jul 07 11:45:14.959505 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:45:14.959227    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcn86\" (UniqueName: \"kubernetes.io/projected/2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e-kube-api-access-xcn86\") pod \"maas-api-e2e-shared-a-b5d0aa-65cbc9b5f5-g4bkh\" (UID: \"2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e\") " pod="opendatahub/maas-api-e2e-shared-a-b5d0aa-65cbc9b5f5-g4bkh"
Jul 07 11:45:14.959505 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:45:14.959368    2570 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-a-b5d0aa-serving-cert: secret "maas-api-e2e-shared-a-b5d0aa-serving-cert" not found
Jul 07 11:45:14.959505 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:45:14.959435    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e-maas-api-tls podName:2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e nodeName:}" failed. No retries permitted until 2026-07-07 11:45:15.459414889 +0000 UTC m=+2065.169311255 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e-maas-api-tls") pod "maas-api-e2e-shared-a-b5d0aa-65cbc9b5f5-g4bkh" (UID: "2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e") : secret "maas-api-e2e-shared-a-b5d0aa-serving-cert" not found
Jul 07 11:45:14.959995 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:45:14.959967    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e-openshift-service-ca\") pod \"maas-api-e2e-shared-a-b5d0aa-65cbc9b5f5-g4bkh\" (UID: \"2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e\") " pod="opendatahub/maas-api-e2e-shared-a-b5d0aa-65cbc9b5f5-g4bkh"
Jul 07 11:45:14.971036 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:45:14.971005    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcn86\" (UniqueName: \"kubernetes.io/projected/2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e-kube-api-access-xcn86\") pod \"maas-api-e2e-shared-a-b5d0aa-65cbc9b5f5-g4bkh\" (UID: \"2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e\") " pod="opendatahub/maas-api-e2e-shared-a-b5d0aa-65cbc9b5f5-g4bkh"
Jul 07 11:45:15.464125 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:45:15.464084    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e-maas-api-tls\") pod \"maas-api-e2e-shared-a-b5d0aa-65cbc9b5f5-g4bkh\" (UID: \"2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e\") " pod="opendatahub/maas-api-e2e-shared-a-b5d0aa-65cbc9b5f5-g4bkh"
Jul 07 11:45:15.466777 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:45:15.466750    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e-maas-api-tls\") pod \"maas-api-e2e-shared-a-b5d0aa-65cbc9b5f5-g4bkh\" (UID: \"2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e\") " pod="opendatahub/maas-api-e2e-shared-a-b5d0aa-65cbc9b5f5-g4bkh"
Jul 07 11:45:15.645399 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:45:15.645361    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-b5d0aa-65cbc9b5f5-g4bkh"
Jul 07 11:45:15.793264 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:45:15.793239    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-b5d0aa-65cbc9b5f5-g4bkh"]
Jul 07 11:45:15.795221 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:45:15.795189    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cfc83d1_5d8c_4cf1_a429_bbf08fb5b62e.slice/crio-cafe44ff06806cdfd0e2e85a45e151a5b5f965af54f101b553b7b1ec4f42f254 WatchSource:0}: Error finding container cafe44ff06806cdfd0e2e85a45e151a5b5f965af54f101b553b7b1ec4f42f254: Status 404 returned error can't find the container with id cafe44ff06806cdfd0e2e85a45e151a5b5f965af54f101b553b7b1ec4f42f254
Jul 07 11:45:16.684133 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:45:16.684095    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-b5d0aa-65cbc9b5f5-g4bkh" event={"ID":"2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e","Type":"ContainerStarted","Data":"fcbba72f6b249765559376e13d815ee242376f4b3d9d559b7b0154ab1c2123b9"}
Jul 07 11:45:16.684302 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:45:16.684179    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-a-b5d0aa-65cbc9b5f5-g4bkh"
Jul 07 11:45:16.684302 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:45:16.684197    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-b5d0aa-65cbc9b5f5-g4bkh" event={"ID":"2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e","Type":"ContainerStarted","Data":"cafe44ff06806cdfd0e2e85a45e151a5b5f965af54f101b553b7b1ec4f42f254"}
Jul 07 11:45:16.705452 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:45:16.705394    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-a-b5d0aa-65cbc9b5f5-g4bkh" podStartSLOduration=2.398032255 podStartE2EDuration="2.705375958s" totalImagesPullingTime="307.343703ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:45:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:45:16.70361346 +0000 UTC m=+2066.413509848" watchObservedRunningTime="2026-07-07 11:45:16.705375958 +0000 UTC m=+2066.415272345"
Jul 07 11:45:22.695985 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:45:22.695952    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-a-b5d0aa-65cbc9b5f5-g4bkh"
Jul 07 11:45:50.894612 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:45:50.894582    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4v7v8_c8c17c90-3e07-4530-8411-d59cef9a1aff/console-operator/2.log"
Jul 07 11:45:50.896501 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:45:50.896476    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4v7v8_c8c17c90-3e07-4530-8411-d59cef9a1aff/console-operator/2.log"
Jul 07 11:46:17.565260 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:46:17.565226    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-43ebe2/auth-test-model-43ebe2-kserve-547dd99d59-sbx9b"]
Jul 07 11:46:17.568852 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:46:17.568836    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-43ebe2/auth-test-model-43ebe2-kserve-547dd99d59-sbx9b"
Jul 07 11:46:17.574174 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:46:17.574151    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-43ebe2\"/\"openshift-service-ca.crt\""
Jul 07 11:46:17.574317 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:46:17.574154    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-43ebe2\"/\"kube-root-ca.crt\""
Jul 07 11:46:17.576445 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:46:17.576425    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-43ebe2\"/\"default-dockercfg-8dc6c\""
Jul 07 11:46:17.576552 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:46:17.576494    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-43ebe2\"/\"auth-test-model-43ebe2-kserve-self-signed-certs\""
Jul 07 11:46:17.587986 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:46:17.587964    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-43ebe2/auth-test-model-43ebe2-kserve-547dd99d59-sbx9b"]
Jul 07 11:46:17.607436 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:46:17.607409    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/42fafae1-1fb8-45d8-9e5e-83b8f48701f9-model-cache\") pod \"auth-test-model-43ebe2-kserve-547dd99d59-sbx9b\" (UID: \"42fafae1-1fb8-45d8-9e5e-83b8f48701f9\") " pod="ai-tenant-e2e-shared-b-43ebe2/auth-test-model-43ebe2-kserve-547dd99d59-sbx9b"
Jul 07 11:46:17.607587 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:46:17.607448    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/42fafae1-1fb8-45d8-9e5e-83b8f48701f9-home\") pod \"auth-test-model-43ebe2-kserve-547dd99d59-sbx9b\" (UID: \"42fafae1-1fb8-45d8-9e5e-83b8f48701f9\") " pod="ai-tenant-e2e-shared-b-43ebe2/auth-test-model-43ebe2-kserve-547dd99d59-sbx9b"
Jul 07 11:46:17.607587 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:46:17.607469    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/42fafae1-1fb8-45d8-9e5e-83b8f48701f9-tls-certs\") pod \"auth-test-model-43ebe2-kserve-547dd99d59-sbx9b\" (UID: \"42fafae1-1fb8-45d8-9e5e-83b8f48701f9\") " pod="ai-tenant-e2e-shared-b-43ebe2/auth-test-model-43ebe2-kserve-547dd99d59-sbx9b"
Jul 07 11:46:17.607587 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:46:17.607537    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42fafae1-1fb8-45d8-9e5e-83b8f48701f9-tmp-dir\") pod \"auth-test-model-43ebe2-kserve-547dd99d59-sbx9b\" (UID: \"42fafae1-1fb8-45d8-9e5e-83b8f48701f9\") " pod="ai-tenant-e2e-shared-b-43ebe2/auth-test-model-43ebe2-kserve-547dd99d59-sbx9b"
Jul 07 11:46:17.607587 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:46:17.607580    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/42fafae1-1fb8-45d8-9e5e-83b8f48701f9-dshm\") pod \"auth-test-model-43ebe2-kserve-547dd99d59-sbx9b\" (UID: \"42fafae1-1fb8-45d8-9e5e-83b8f48701f9\") " pod="ai-tenant-e2e-shared-b-43ebe2/auth-test-model-43ebe2-kserve-547dd99d59-sbx9b"
Jul 07 11:46:17.607752 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:46:17.607600    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nj96p\" (UniqueName: \"kubernetes.io/projected/42fafae1-1fb8-45d8-9e5e-83b8f48701f9-kube-api-access-nj96p\") pod \"auth-test-model-43ebe2-kserve-547dd99d59-sbx9b\" (UID: \"42fafae1-1fb8-45d8-9e5e-83b8f48701f9\") " pod="ai-tenant-e2e-shared-b-43ebe2/auth-test-model-43ebe2-kserve-547dd99d59-sbx9b"
Jul 07 11:46:17.708095 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:46:17.708058    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/42fafae1-1fb8-45d8-9e5e-83b8f48701f9-model-cache\") pod \"auth-test-model-43ebe2-kserve-547dd99d59-sbx9b\" (UID: \"42fafae1-1fb8-45d8-9e5e-83b8f48701f9\") " pod="ai-tenant-e2e-shared-b-43ebe2/auth-test-model-43ebe2-kserve-547dd99d59-sbx9b"
Jul 07 11:46:17.708289 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:46:17.708105    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/42fafae1-1fb8-45d8-9e5e-83b8f48701f9-home\") pod \"auth-test-model-43ebe2-kserve-547dd99d59-sbx9b\" (UID: \"42fafae1-1fb8-45d8-9e5e-83b8f48701f9\") " pod="ai-tenant-e2e-shared-b-43ebe2/auth-test-model-43ebe2-kserve-547dd99d59-sbx9b"
Jul 07 11:46:17.708289 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:46:17.708125    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/42fafae1-1fb8-45d8-9e5e-83b8f48701f9-tls-certs\") pod \"auth-test-model-43ebe2-kserve-547dd99d59-sbx9b\" (UID: \"42fafae1-1fb8-45d8-9e5e-83b8f48701f9\") " pod="ai-tenant-e2e-shared-b-43ebe2/auth-test-model-43ebe2-kserve-547dd99d59-sbx9b"
Jul 07 11:46:17.708289 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:46:17.708147    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42fafae1-1fb8-45d8-9e5e-83b8f48701f9-tmp-dir\") pod \"auth-test-model-43ebe2-kserve-547dd99d59-sbx9b\" (UID: \"42fafae1-1fb8-45d8-9e5e-83b8f48701f9\") " pod="ai-tenant-e2e-shared-b-43ebe2/auth-test-model-43ebe2-kserve-547dd99d59-sbx9b"
Jul 07 11:46:17.708289 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:46:17.708164    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/42fafae1-1fb8-45d8-9e5e-83b8f48701f9-dshm\") pod \"auth-test-model-43ebe2-kserve-547dd99d59-sbx9b\" (UID: \"42fafae1-1fb8-45d8-9e5e-83b8f48701f9\") " pod="ai-tenant-e2e-shared-b-43ebe2/auth-test-model-43ebe2-kserve-547dd99d59-sbx9b"
Jul 07 11:46:17.708289 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:46:17.708182    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nj96p\" (UniqueName: \"kubernetes.io/projected/42fafae1-1fb8-45d8-9e5e-83b8f48701f9-kube-api-access-nj96p\") pod \"auth-test-model-43ebe2-kserve-547dd99d59-sbx9b\" (UID: \"42fafae1-1fb8-45d8-9e5e-83b8f48701f9\") " pod="ai-tenant-e2e-shared-b-43ebe2/auth-test-model-43ebe2-kserve-547dd99d59-sbx9b"
Jul 07 11:46:17.708601 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:46:17.708560    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42fafae1-1fb8-45d8-9e5e-83b8f48701f9-tmp-dir\") pod \"auth-test-model-43ebe2-kserve-547dd99d59-sbx9b\" (UID: \"42fafae1-1fb8-45d8-9e5e-83b8f48701f9\") " pod="ai-tenant-e2e-shared-b-43ebe2/auth-test-model-43ebe2-kserve-547dd99d59-sbx9b"
Jul 07 11:46:17.708601 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:46:17.708574    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/42fafae1-1fb8-45d8-9e5e-83b8f48701f9-home\") pod \"auth-test-model-43ebe2-kserve-547dd99d59-sbx9b\" (UID: \"42fafae1-1fb8-45d8-9e5e-83b8f48701f9\") " pod="ai-tenant-e2e-shared-b-43ebe2/auth-test-model-43ebe2-kserve-547dd99d59-sbx9b"
Jul 07 11:46:17.708683 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:46:17.708608    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/42fafae1-1fb8-45d8-9e5e-83b8f48701f9-model-cache\") pod \"auth-test-model-43ebe2-kserve-547dd99d59-sbx9b\" (UID: \"42fafae1-1fb8-45d8-9e5e-83b8f48701f9\") " pod="ai-tenant-e2e-shared-b-43ebe2/auth-test-model-43ebe2-kserve-547dd99d59-sbx9b"
Jul 07 11:46:17.710617 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:46:17.710594    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/42fafae1-1fb8-45d8-9e5e-83b8f48701f9-dshm\") pod \"auth-test-model-43ebe2-kserve-547dd99d59-sbx9b\" (UID: \"42fafae1-1fb8-45d8-9e5e-83b8f48701f9\") " pod="ai-tenant-e2e-shared-b-43ebe2/auth-test-model-43ebe2-kserve-547dd99d59-sbx9b"
Jul 07 11:46:17.710988 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:46:17.710969    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/42fafae1-1fb8-45d8-9e5e-83b8f48701f9-tls-certs\") pod \"auth-test-model-43ebe2-kserve-547dd99d59-sbx9b\" (UID: \"42fafae1-1fb8-45d8-9e5e-83b8f48701f9\") " pod="ai-tenant-e2e-shared-b-43ebe2/auth-test-model-43ebe2-kserve-547dd99d59-sbx9b"
Jul 07 11:46:17.720570 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:46:17.720547    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nj96p\" (UniqueName: \"kubernetes.io/projected/42fafae1-1fb8-45d8-9e5e-83b8f48701f9-kube-api-access-nj96p\") pod \"auth-test-model-43ebe2-kserve-547dd99d59-sbx9b\" (UID: \"42fafae1-1fb8-45d8-9e5e-83b8f48701f9\") " pod="ai-tenant-e2e-shared-b-43ebe2/auth-test-model-43ebe2-kserve-547dd99d59-sbx9b"
Jul 07 11:46:17.878388 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:46:17.878293    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-43ebe2/auth-test-model-43ebe2-kserve-547dd99d59-sbx9b"
Jul 07 11:46:18.018524 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:46:18.018499    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-43ebe2/auth-test-model-43ebe2-kserve-547dd99d59-sbx9b"]
Jul 07 11:46:18.020756 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:46:18.020724    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod42fafae1_1fb8_45d8_9e5e_83b8f48701f9.slice/crio-221aeb37d49a5b46049b2bf43d12cf69677f01b2cefcc95e97afe7dfb85c8d41 WatchSource:0}: Error finding container 221aeb37d49a5b46049b2bf43d12cf69677f01b2cefcc95e97afe7dfb85c8d41: Status 404 returned error can't find the container with id 221aeb37d49a5b46049b2bf43d12cf69677f01b2cefcc95e97afe7dfb85c8d41
Jul 07 11:46:18.991563 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:46:18.991532    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-43ebe2/auth-test-model-43ebe2-kserve-547dd99d59-sbx9b" event={"ID":"42fafae1-1fb8-45d8-9e5e-83b8f48701f9","Type":"ContainerStarted","Data":"a0bb461e2baa2cf09871a1f456b371b728f9624c3e25a32f88fff4672da9eef1"}
Jul 07 11:46:18.991955 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:46:18.991573    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-43ebe2/auth-test-model-43ebe2-kserve-547dd99d59-sbx9b" event={"ID":"42fafae1-1fb8-45d8-9e5e-83b8f48701f9","Type":"ContainerStarted","Data":"221aeb37d49a5b46049b2bf43d12cf69677f01b2cefcc95e97afe7dfb85c8d41"}
Jul 07 11:46:19.011593 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:46:19.011542    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-43ebe2/auth-test-model-43ebe2-kserve-547dd99d59-sbx9b" podStartSLOduration=2.011526406 podStartE2EDuration="2.011526406s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:46:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:46:19.009902207 +0000 UTC m=+2128.719798595" watchObservedRunningTime="2026-07-07 11:46:19.011526406 +0000 UTC m=+2128.721422817"
Jul 07 11:46:27.879274 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:46:27.879223    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-43ebe2/auth-test-model-43ebe2-kserve-547dd99d59-sbx9b"
Jul 07 11:46:27.879274 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:46:27.879279    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-43ebe2/auth-test-model-43ebe2-kserve-547dd99d59-sbx9b"
Jul 07 11:46:27.891707 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:46:27.891682    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-43ebe2/auth-test-model-43ebe2-kserve-547dd99d59-sbx9b"
Jul 07 11:46:28.046901 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:46:28.046866    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-43ebe2/auth-test-model-43ebe2-kserve-547dd99d59-sbx9b"
Jul 07 11:48:10.613126 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:10.613093    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-43ebe2/auth-test-model-43ebe2-kserve-547dd99d59-sbx9b"]
Jul 07 11:48:15.132270 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:15.132196    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-b5d0aa/sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp"]
Jul 07 11:48:15.135638 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:15.135622    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b5d0aa/sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp"
Jul 07 11:48:15.138673 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:15.138653    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-b5d0aa\"/\"kube-root-ca.crt\""
Jul 07 11:48:15.140151 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:15.140119    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b5d0aa\"/\"default-dockercfg-9ltmv\""
Jul 07 11:48:15.140151 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:15.140119    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b5d0aa\"/\"sub-test-model-b5d0aa-kserve-self-signed-certs\""
Jul 07 11:48:15.140345 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:15.140178    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-b5d0aa\"/\"openshift-service-ca.crt\""
Jul 07 11:48:15.148875 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:15.148849    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b5d0aa/sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp"]
Jul 07 11:48:15.220199 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:15.220160    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba5bfdf6-54a5-4087-b882-9fd51391cb90-tmp-dir\") pod \"sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp\" (UID: \"ba5bfdf6-54a5-4087-b882-9fd51391cb90\") " pod="ai-tenant-e2e-shared-a-b5d0aa/sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp"
Jul 07 11:48:15.220419 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:15.220242    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba5bfdf6-54a5-4087-b882-9fd51391cb90-model-cache\") pod \"sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp\" (UID: \"ba5bfdf6-54a5-4087-b882-9fd51391cb90\") " pod="ai-tenant-e2e-shared-a-b5d0aa/sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp"
Jul 07 11:48:15.220419 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:15.220271    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba5bfdf6-54a5-4087-b882-9fd51391cb90-dshm\") pod \"sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp\" (UID: \"ba5bfdf6-54a5-4087-b882-9fd51391cb90\") " pod="ai-tenant-e2e-shared-a-b5d0aa/sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp"
Jul 07 11:48:15.220419 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:15.220295    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba5bfdf6-54a5-4087-b882-9fd51391cb90-tls-certs\") pod \"sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp\" (UID: \"ba5bfdf6-54a5-4087-b882-9fd51391cb90\") " pod="ai-tenant-e2e-shared-a-b5d0aa/sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp"
Jul 07 11:48:15.220419 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:15.220391    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba5bfdf6-54a5-4087-b882-9fd51391cb90-home\") pod \"sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp\" (UID: \"ba5bfdf6-54a5-4087-b882-9fd51391cb90\") " pod="ai-tenant-e2e-shared-a-b5d0aa/sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp"
Jul 07 11:48:15.220594 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:15.220418    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9z24z\" (UniqueName: \"kubernetes.io/projected/ba5bfdf6-54a5-4087-b882-9fd51391cb90-kube-api-access-9z24z\") pod \"sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp\" (UID: \"ba5bfdf6-54a5-4087-b882-9fd51391cb90\") " pod="ai-tenant-e2e-shared-a-b5d0aa/sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp"
Jul 07 11:48:15.321434 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:15.321390    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba5bfdf6-54a5-4087-b882-9fd51391cb90-model-cache\") pod \"sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp\" (UID: \"ba5bfdf6-54a5-4087-b882-9fd51391cb90\") " pod="ai-tenant-e2e-shared-a-b5d0aa/sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp"
Jul 07 11:48:15.321434 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:15.321435    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba5bfdf6-54a5-4087-b882-9fd51391cb90-dshm\") pod \"sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp\" (UID: \"ba5bfdf6-54a5-4087-b882-9fd51391cb90\") " pod="ai-tenant-e2e-shared-a-b5d0aa/sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp"
Jul 07 11:48:15.321683 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:15.321451    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba5bfdf6-54a5-4087-b882-9fd51391cb90-tls-certs\") pod \"sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp\" (UID: \"ba5bfdf6-54a5-4087-b882-9fd51391cb90\") " pod="ai-tenant-e2e-shared-a-b5d0aa/sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp"
Jul 07 11:48:15.321683 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:15.321481    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba5bfdf6-54a5-4087-b882-9fd51391cb90-home\") pod \"sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp\" (UID: \"ba5bfdf6-54a5-4087-b882-9fd51391cb90\") " pod="ai-tenant-e2e-shared-a-b5d0aa/sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp"
Jul 07 11:48:15.321683 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:15.321504    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9z24z\" (UniqueName: \"kubernetes.io/projected/ba5bfdf6-54a5-4087-b882-9fd51391cb90-kube-api-access-9z24z\") pod \"sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp\" (UID: \"ba5bfdf6-54a5-4087-b882-9fd51391cb90\") " pod="ai-tenant-e2e-shared-a-b5d0aa/sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp"
Jul 07 11:48:15.321683 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:15.321571    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba5bfdf6-54a5-4087-b882-9fd51391cb90-tmp-dir\") pod \"sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp\" (UID: \"ba5bfdf6-54a5-4087-b882-9fd51391cb90\") " pod="ai-tenant-e2e-shared-a-b5d0aa/sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp"
Jul 07 11:48:15.321949 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:15.321923    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba5bfdf6-54a5-4087-b882-9fd51391cb90-model-cache\") pod \"sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp\" (UID: \"ba5bfdf6-54a5-4087-b882-9fd51391cb90\") " pod="ai-tenant-e2e-shared-a-b5d0aa/sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp"
Jul 07 11:48:15.322078 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:15.321929    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba5bfdf6-54a5-4087-b882-9fd51391cb90-home\") pod \"sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp\" (UID: \"ba5bfdf6-54a5-4087-b882-9fd51391cb90\") " pod="ai-tenant-e2e-shared-a-b5d0aa/sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp"
Jul 07 11:48:15.322144 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:15.322123    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba5bfdf6-54a5-4087-b882-9fd51391cb90-tmp-dir\") pod \"sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp\" (UID: \"ba5bfdf6-54a5-4087-b882-9fd51391cb90\") " pod="ai-tenant-e2e-shared-a-b5d0aa/sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp"
Jul 07 11:48:15.324055 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:15.324032    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba5bfdf6-54a5-4087-b882-9fd51391cb90-dshm\") pod \"sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp\" (UID: \"ba5bfdf6-54a5-4087-b882-9fd51391cb90\") " pod="ai-tenant-e2e-shared-a-b5d0aa/sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp"
Jul 07 11:48:15.324267 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:15.324247    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba5bfdf6-54a5-4087-b882-9fd51391cb90-tls-certs\") pod \"sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp\" (UID: \"ba5bfdf6-54a5-4087-b882-9fd51391cb90\") " pod="ai-tenant-e2e-shared-a-b5d0aa/sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp"
Jul 07 11:48:15.343423 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:15.343392    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9z24z\" (UniqueName: \"kubernetes.io/projected/ba5bfdf6-54a5-4087-b882-9fd51391cb90-kube-api-access-9z24z\") pod \"sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp\" (UID: \"ba5bfdf6-54a5-4087-b882-9fd51391cb90\") " pod="ai-tenant-e2e-shared-a-b5d0aa/sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp"
Jul 07 11:48:15.444595 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:15.444503    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b5d0aa/sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp"
Jul 07 11:48:15.597978 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:15.597955    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b5d0aa/sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp"]
Jul 07 11:48:15.600417 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:48:15.600390    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podba5bfdf6_54a5_4087_b882_9fd51391cb90.slice/crio-7a1dad61c7b58e5bd931843cf4788cbb4b22d19edcc7e8a533a85db4ad96fbc7 WatchSource:0}: Error finding container 7a1dad61c7b58e5bd931843cf4788cbb4b22d19edcc7e8a533a85db4ad96fbc7: Status 404 returned error can't find the container with id 7a1dad61c7b58e5bd931843cf4788cbb4b22d19edcc7e8a533a85db4ad96fbc7
Jul 07 11:48:16.562374 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:16.562314    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b5d0aa/sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp" event={"ID":"ba5bfdf6-54a5-4087-b882-9fd51391cb90","Type":"ContainerStarted","Data":"084259d551af727f34a854cc0781aea3747ab2650652cf0390fd8002a0fb1b56"}
Jul 07 11:48:16.562760 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:16.562384    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b5d0aa/sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp" event={"ID":"ba5bfdf6-54a5-4087-b882-9fd51391cb90","Type":"ContainerStarted","Data":"7a1dad61c7b58e5bd931843cf4788cbb4b22d19edcc7e8a533a85db4ad96fbc7"}
Jul 07 11:48:16.618722 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:16.618667    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-b5d0aa/sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp" podStartSLOduration=1.618650396 podStartE2EDuration="1.618650396s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:48:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:48:16.617759913 +0000 UTC m=+2246.327656302" watchObservedRunningTime="2026-07-07 11:48:16.618650396 +0000 UTC m=+2246.328546782"
Jul 07 11:48:25.444920 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:25.444875    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-b5d0aa/sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp"
Jul 07 11:48:25.445316 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:25.444936    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-b5d0aa/sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp"
Jul 07 11:48:25.457515 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:25.457488    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-b5d0aa/sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp"
Jul 07 11:48:25.617184 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:25.617155    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-b5d0aa/sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp"
Jul 07 11:48:25.633805 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:25.633770    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-43ebe2/auth-test-model-43ebe2-kserve-547dd99d59-sbx9b" podUID="42fafae1-1fb8-45d8-9e5e-83b8f48701f9" containerName="main" containerID="cri-o://a0bb461e2baa2cf09871a1f456b371b728f9624c3e25a32f88fff4672da9eef1" gracePeriod=45
Jul 07 11:48:25.882462 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:25.882437    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-43ebe2/auth-test-model-43ebe2-kserve-547dd99d59-sbx9b"
Jul 07 11:48:25.928575 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:25.928547    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/42fafae1-1fb8-45d8-9e5e-83b8f48701f9-tls-certs\") pod \"42fafae1-1fb8-45d8-9e5e-83b8f48701f9\" (UID: \"42fafae1-1fb8-45d8-9e5e-83b8f48701f9\") "
Jul 07 11:48:25.928753 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:25.928612    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/42fafae1-1fb8-45d8-9e5e-83b8f48701f9-home\") pod \"42fafae1-1fb8-45d8-9e5e-83b8f48701f9\" (UID: \"42fafae1-1fb8-45d8-9e5e-83b8f48701f9\") "
Jul 07 11:48:25.928753 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:25.928667    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nj96p\" (UniqueName: \"kubernetes.io/projected/42fafae1-1fb8-45d8-9e5e-83b8f48701f9-kube-api-access-nj96p\") pod \"42fafae1-1fb8-45d8-9e5e-83b8f48701f9\" (UID: \"42fafae1-1fb8-45d8-9e5e-83b8f48701f9\") "
Jul 07 11:48:25.928753 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:25.928721    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/42fafae1-1fb8-45d8-9e5e-83b8f48701f9-dshm\") pod \"42fafae1-1fb8-45d8-9e5e-83b8f48701f9\" (UID: \"42fafae1-1fb8-45d8-9e5e-83b8f48701f9\") "
Jul 07 11:48:25.928920 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:25.928765    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/42fafae1-1fb8-45d8-9e5e-83b8f48701f9-model-cache\") pod \"42fafae1-1fb8-45d8-9e5e-83b8f48701f9\" (UID: \"42fafae1-1fb8-45d8-9e5e-83b8f48701f9\") "
Jul 07 11:48:25.928920 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:25.928790    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42fafae1-1fb8-45d8-9e5e-83b8f48701f9-tmp-dir\") pod \"42fafae1-1fb8-45d8-9e5e-83b8f48701f9\" (UID: \"42fafae1-1fb8-45d8-9e5e-83b8f48701f9\") "
Jul 07 11:48:25.929033 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:25.928917    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42fafae1-1fb8-45d8-9e5e-83b8f48701f9-home" (OuterVolumeSpecName: "home") pod "42fafae1-1fb8-45d8-9e5e-83b8f48701f9" (UID: "42fafae1-1fb8-45d8-9e5e-83b8f48701f9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:48:25.929147 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:25.929121    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42fafae1-1fb8-45d8-9e5e-83b8f48701f9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "42fafae1-1fb8-45d8-9e5e-83b8f48701f9" (UID: "42fafae1-1fb8-45d8-9e5e-83b8f48701f9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:48:25.929223 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:25.929129    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/42fafae1-1fb8-45d8-9e5e-83b8f48701f9-home\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:48:25.929223 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:25.929145    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42fafae1-1fb8-45d8-9e5e-83b8f48701f9-model-cache" (OuterVolumeSpecName: "model-cache") pod "42fafae1-1fb8-45d8-9e5e-83b8f48701f9" (UID: "42fafae1-1fb8-45d8-9e5e-83b8f48701f9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:48:25.931086 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:25.931063    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42fafae1-1fb8-45d8-9e5e-83b8f48701f9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "42fafae1-1fb8-45d8-9e5e-83b8f48701f9" (UID: "42fafae1-1fb8-45d8-9e5e-83b8f48701f9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 11:48:25.931210 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:25.931193    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42fafae1-1fb8-45d8-9e5e-83b8f48701f9-kube-api-access-nj96p" (OuterVolumeSpecName: "kube-api-access-nj96p") pod "42fafae1-1fb8-45d8-9e5e-83b8f48701f9" (UID: "42fafae1-1fb8-45d8-9e5e-83b8f48701f9"). InnerVolumeSpecName "kube-api-access-nj96p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:48:25.931395 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:25.931380    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42fafae1-1fb8-45d8-9e5e-83b8f48701f9-dshm" (OuterVolumeSpecName: "dshm") pod "42fafae1-1fb8-45d8-9e5e-83b8f48701f9" (UID: "42fafae1-1fb8-45d8-9e5e-83b8f48701f9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:48:26.030310 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:26.030210    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/42fafae1-1fb8-45d8-9e5e-83b8f48701f9-model-cache\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:48:26.030310 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:26.030246    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42fafae1-1fb8-45d8-9e5e-83b8f48701f9-tmp-dir\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:48:26.030310 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:26.030257    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/42fafae1-1fb8-45d8-9e5e-83b8f48701f9-tls-certs\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:48:26.030310 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:26.030269    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nj96p\" (UniqueName: \"kubernetes.io/projected/42fafae1-1fb8-45d8-9e5e-83b8f48701f9-kube-api-access-nj96p\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:48:26.030310 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:26.030282    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/42fafae1-1fb8-45d8-9e5e-83b8f48701f9-dshm\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:48:26.612414 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:26.612376    2570 generic.go:356] "Generic (PLEG): container finished" podID="42fafae1-1fb8-45d8-9e5e-83b8f48701f9" containerID="a0bb461e2baa2cf09871a1f456b371b728f9624c3e25a32f88fff4672da9eef1" exitCode=0
Jul 07 11:48:26.612414 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:26.612407    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-43ebe2/auth-test-model-43ebe2-kserve-547dd99d59-sbx9b" event={"ID":"42fafae1-1fb8-45d8-9e5e-83b8f48701f9","Type":"ContainerDied","Data":"a0bb461e2baa2cf09871a1f456b371b728f9624c3e25a32f88fff4672da9eef1"}
Jul 07 11:48:26.612994 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:26.612431    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-43ebe2/auth-test-model-43ebe2-kserve-547dd99d59-sbx9b"
Jul 07 11:48:26.612994 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:26.612447    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-43ebe2/auth-test-model-43ebe2-kserve-547dd99d59-sbx9b" event={"ID":"42fafae1-1fb8-45d8-9e5e-83b8f48701f9","Type":"ContainerDied","Data":"221aeb37d49a5b46049b2bf43d12cf69677f01b2cefcc95e97afe7dfb85c8d41"}
Jul 07 11:48:26.612994 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:26.612467    2570 scope.go:117] "RemoveContainer" containerID="a0bb461e2baa2cf09871a1f456b371b728f9624c3e25a32f88fff4672da9eef1"
Jul 07 11:48:26.622115 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:26.622099    2570 scope.go:117] "RemoveContainer" containerID="a0bb461e2baa2cf09871a1f456b371b728f9624c3e25a32f88fff4672da9eef1"
Jul 07 11:48:26.622388 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:48:26.622369    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0bb461e2baa2cf09871a1f456b371b728f9624c3e25a32f88fff4672da9eef1\": container with ID starting with a0bb461e2baa2cf09871a1f456b371b728f9624c3e25a32f88fff4672da9eef1 not found: ID does not exist" containerID="a0bb461e2baa2cf09871a1f456b371b728f9624c3e25a32f88fff4672da9eef1"
Jul 07 11:48:26.622468 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:26.622396    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0bb461e2baa2cf09871a1f456b371b728f9624c3e25a32f88fff4672da9eef1"} err="failed to get container status \"a0bb461e2baa2cf09871a1f456b371b728f9624c3e25a32f88fff4672da9eef1\": rpc error: code = NotFound desc = could not find container \"a0bb461e2baa2cf09871a1f456b371b728f9624c3e25a32f88fff4672da9eef1\": container with ID starting with a0bb461e2baa2cf09871a1f456b371b728f9624c3e25a32f88fff4672da9eef1 not found: ID does not exist"
Jul 07 11:48:26.636898 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:26.636872    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-43ebe2/auth-test-model-43ebe2-kserve-547dd99d59-sbx9b"]
Jul 07 11:48:26.639861 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:26.639840    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-43ebe2/auth-test-model-43ebe2-kserve-547dd99d59-sbx9b"]
Jul 07 11:48:26.822167 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:26.822134    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42fafae1-1fb8-45d8-9e5e-83b8f48701f9" path="/var/lib/kubelet/pods/42fafae1-1fb8-45d8-9e5e-83b8f48701f9/volumes"
Jul 07 11:48:51.029923 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:48:51.029891    2570 scope.go:117] "RemoveContainer" containerID="e3f33ca48c719c28130efb04d306f494de894196796333fc70274c15e6ec4c2c"
Jul 07 11:49:48.760674 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:49:48.760583    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b5d0aa/sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp"]
Jul 07 11:49:52.424887 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:49:52.424856    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-b5d0aa/rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7"]
Jul 07 11:49:52.425286 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:49:52.425237    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42fafae1-1fb8-45d8-9e5e-83b8f48701f9" containerName="main"
Jul 07 11:49:52.425286 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:49:52.425248    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="42fafae1-1fb8-45d8-9e5e-83b8f48701f9" containerName="main"
Jul 07 11:49:52.425395 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:49:52.425303    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="42fafae1-1fb8-45d8-9e5e-83b8f48701f9" containerName="main"
Jul 07 11:49:52.428641 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:49:52.428620    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b5d0aa/rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7"
Jul 07 11:49:52.431748 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:49:52.431725    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b5d0aa\"/\"rate-test-model-b5d0aa-kserve-self-signed-certs\""
Jul 07 11:49:52.438284 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:49:52.438253    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b5d0aa/rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7"]
Jul 07 11:49:52.520414 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:49:52.520378    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/978a6abb-16d0-41a9-b4cd-1f16e8ff7a67-model-cache\") pod \"rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7\" (UID: \"978a6abb-16d0-41a9-b4cd-1f16e8ff7a67\") " pod="ai-tenant-e2e-shared-a-b5d0aa/rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7"
Jul 07 11:49:52.520414 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:49:52.520415    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/978a6abb-16d0-41a9-b4cd-1f16e8ff7a67-tmp-dir\") pod \"rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7\" (UID: \"978a6abb-16d0-41a9-b4cd-1f16e8ff7a67\") " pod="ai-tenant-e2e-shared-a-b5d0aa/rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7"
Jul 07 11:49:52.520659 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:49:52.520454    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rq5x\" (UniqueName: \"kubernetes.io/projected/978a6abb-16d0-41a9-b4cd-1f16e8ff7a67-kube-api-access-6rq5x\") pod \"rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7\" (UID: \"978a6abb-16d0-41a9-b4cd-1f16e8ff7a67\") " pod="ai-tenant-e2e-shared-a-b5d0aa/rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7"
Jul 07 11:49:52.520659 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:49:52.520527    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/978a6abb-16d0-41a9-b4cd-1f16e8ff7a67-tls-certs\") pod \"rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7\" (UID: \"978a6abb-16d0-41a9-b4cd-1f16e8ff7a67\") " pod="ai-tenant-e2e-shared-a-b5d0aa/rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7"
Jul 07 11:49:52.520659 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:49:52.520566    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/978a6abb-16d0-41a9-b4cd-1f16e8ff7a67-home\") pod \"rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7\" (UID: \"978a6abb-16d0-41a9-b4cd-1f16e8ff7a67\") " pod="ai-tenant-e2e-shared-a-b5d0aa/rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7"
Jul 07 11:49:52.520659 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:49:52.520587    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/978a6abb-16d0-41a9-b4cd-1f16e8ff7a67-dshm\") pod \"rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7\" (UID: \"978a6abb-16d0-41a9-b4cd-1f16e8ff7a67\") " pod="ai-tenant-e2e-shared-a-b5d0aa/rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7"
Jul 07 11:49:52.621351 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:49:52.621297    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rq5x\" (UniqueName: \"kubernetes.io/projected/978a6abb-16d0-41a9-b4cd-1f16e8ff7a67-kube-api-access-6rq5x\") pod \"rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7\" (UID: \"978a6abb-16d0-41a9-b4cd-1f16e8ff7a67\") " pod="ai-tenant-e2e-shared-a-b5d0aa/rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7"
Jul 07 11:49:52.621559 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:49:52.621359    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/978a6abb-16d0-41a9-b4cd-1f16e8ff7a67-tls-certs\") pod \"rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7\" (UID: \"978a6abb-16d0-41a9-b4cd-1f16e8ff7a67\") " pod="ai-tenant-e2e-shared-a-b5d0aa/rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7"
Jul 07 11:49:52.621559 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:49:52.621414    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/978a6abb-16d0-41a9-b4cd-1f16e8ff7a67-home\") pod \"rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7\" (UID: \"978a6abb-16d0-41a9-b4cd-1f16e8ff7a67\") " pod="ai-tenant-e2e-shared-a-b5d0aa/rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7"
Jul 07 11:49:52.621559 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:49:52.621433    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/978a6abb-16d0-41a9-b4cd-1f16e8ff7a67-dshm\") pod \"rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7\" (UID: \"978a6abb-16d0-41a9-b4cd-1f16e8ff7a67\") " pod="ai-tenant-e2e-shared-a-b5d0aa/rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7"
Jul 07 11:49:52.621559 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:49:52.621500    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/978a6abb-16d0-41a9-b4cd-1f16e8ff7a67-model-cache\") pod \"rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7\" (UID: \"978a6abb-16d0-41a9-b4cd-1f16e8ff7a67\") " pod="ai-tenant-e2e-shared-a-b5d0aa/rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7"
Jul 07 11:49:52.621559 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:49:52.621525    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/978a6abb-16d0-41a9-b4cd-1f16e8ff7a67-tmp-dir\") pod \"rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7\" (UID: \"978a6abb-16d0-41a9-b4cd-1f16e8ff7a67\") " pod="ai-tenant-e2e-shared-a-b5d0aa/rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7"
Jul 07 11:49:52.621837 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:49:52.621813    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/978a6abb-16d0-41a9-b4cd-1f16e8ff7a67-home\") pod \"rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7\" (UID: \"978a6abb-16d0-41a9-b4cd-1f16e8ff7a67\") " pod="ai-tenant-e2e-shared-a-b5d0aa/rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7"
Jul 07 11:49:52.621903 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:49:52.621877    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/978a6abb-16d0-41a9-b4cd-1f16e8ff7a67-tmp-dir\") pod \"rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7\" (UID: \"978a6abb-16d0-41a9-b4cd-1f16e8ff7a67\") " pod="ai-tenant-e2e-shared-a-b5d0aa/rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7"
Jul 07 11:49:52.621903 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:49:52.621893    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/978a6abb-16d0-41a9-b4cd-1f16e8ff7a67-model-cache\") pod \"rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7\" (UID: \"978a6abb-16d0-41a9-b4cd-1f16e8ff7a67\") " pod="ai-tenant-e2e-shared-a-b5d0aa/rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7"
Jul 07 11:49:52.623918 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:49:52.623892    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/978a6abb-16d0-41a9-b4cd-1f16e8ff7a67-dshm\") pod \"rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7\" (UID: \"978a6abb-16d0-41a9-b4cd-1f16e8ff7a67\") " pod="ai-tenant-e2e-shared-a-b5d0aa/rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7"
Jul 07 11:49:52.624142 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:49:52.624125    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/978a6abb-16d0-41a9-b4cd-1f16e8ff7a67-tls-certs\") pod \"rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7\" (UID: \"978a6abb-16d0-41a9-b4cd-1f16e8ff7a67\") " pod="ai-tenant-e2e-shared-a-b5d0aa/rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7"
Jul 07 11:49:52.634950 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:49:52.634920    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rq5x\" (UniqueName: \"kubernetes.io/projected/978a6abb-16d0-41a9-b4cd-1f16e8ff7a67-kube-api-access-6rq5x\") pod \"rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7\" (UID: \"978a6abb-16d0-41a9-b4cd-1f16e8ff7a67\") " pod="ai-tenant-e2e-shared-a-b5d0aa/rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7"
Jul 07 11:49:52.738657 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:49:52.738626    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b5d0aa/rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7"
Jul 07 11:49:52.910036 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:49:52.910002    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b5d0aa/rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7"]
Jul 07 11:49:52.912582 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:49:52.912550    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod978a6abb_16d0_41a9_b4cd_1f16e8ff7a67.slice/crio-a7590eee7fba867edf27510eaee66f466212114fe369dd2592a3b5f2a609ab46 WatchSource:0}: Error finding container a7590eee7fba867edf27510eaee66f466212114fe369dd2592a3b5f2a609ab46: Status 404 returned error can't find the container with id a7590eee7fba867edf27510eaee66f466212114fe369dd2592a3b5f2a609ab46
Jul 07 11:49:53.035882 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:49:53.035779    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b5d0aa/rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7" event={"ID":"978a6abb-16d0-41a9-b4cd-1f16e8ff7a67","Type":"ContainerStarted","Data":"a3d37cfcc6f0a5112edff9f945dfa37c913a06118368f37240ed31347ce6f303"}
Jul 07 11:49:53.035882 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:49:53.035836    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b5d0aa/rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7" event={"ID":"978a6abb-16d0-41a9-b4cd-1f16e8ff7a67","Type":"ContainerStarted","Data":"a7590eee7fba867edf27510eaee66f466212114fe369dd2592a3b5f2a609ab46"}
Jul 07 11:49:53.057512 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:49:53.057453    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-b5d0aa/rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7" podStartSLOduration=1.057438508 podStartE2EDuration="1.057438508s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:49:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:49:53.056259304 +0000 UTC m=+2342.766155692" watchObservedRunningTime="2026-07-07 11:49:53.057438508 +0000 UTC m=+2342.767334921"
Jul 07 11:50:02.739901 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:02.739856    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-b5d0aa/rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7"
Jul 07 11:50:02.740505 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:02.740123    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-b5d0aa/rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7"
Jul 07 11:50:02.752703 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:02.752675    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-b5d0aa/rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7"
Jul 07 11:50:03.095462 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:03.095374    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-b5d0aa/rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7"
Jul 07 11:50:03.781035 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:03.780982    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-b5d0aa/sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp" podUID="ba5bfdf6-54a5-4087-b882-9fd51391cb90" containerName="main" containerID="cri-o://084259d551af727f34a854cc0781aea3747ab2650652cf0390fd8002a0fb1b56" gracePeriod=45
Jul 07 11:50:04.047840 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:04.047811    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b5d0aa/sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp"
Jul 07 11:50:04.089173 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:04.089138    2570 generic.go:356] "Generic (PLEG): container finished" podID="ba5bfdf6-54a5-4087-b882-9fd51391cb90" containerID="084259d551af727f34a854cc0781aea3747ab2650652cf0390fd8002a0fb1b56" exitCode=0
Jul 07 11:50:04.089368 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:04.089182    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b5d0aa/sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp"
Jul 07 11:50:04.089368 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:04.089222    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b5d0aa/sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp" event={"ID":"ba5bfdf6-54a5-4087-b882-9fd51391cb90","Type":"ContainerDied","Data":"084259d551af727f34a854cc0781aea3747ab2650652cf0390fd8002a0fb1b56"}
Jul 07 11:50:04.089368 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:04.089272    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b5d0aa/sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp" event={"ID":"ba5bfdf6-54a5-4087-b882-9fd51391cb90","Type":"ContainerDied","Data":"7a1dad61c7b58e5bd931843cf4788cbb4b22d19edcc7e8a533a85db4ad96fbc7"}
Jul 07 11:50:04.089368 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:04.089289    2570 scope.go:117] "RemoveContainer" containerID="084259d551af727f34a854cc0781aea3747ab2650652cf0390fd8002a0fb1b56"
Jul 07 11:50:04.098521 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:04.098499    2570 scope.go:117] "RemoveContainer" containerID="084259d551af727f34a854cc0781aea3747ab2650652cf0390fd8002a0fb1b56"
Jul 07 11:50:04.098848 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:50:04.098815    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"084259d551af727f34a854cc0781aea3747ab2650652cf0390fd8002a0fb1b56\": container with ID starting with 084259d551af727f34a854cc0781aea3747ab2650652cf0390fd8002a0fb1b56 not found: ID does not exist" containerID="084259d551af727f34a854cc0781aea3747ab2650652cf0390fd8002a0fb1b56"
Jul 07 11:50:04.098932 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:04.098858    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"084259d551af727f34a854cc0781aea3747ab2650652cf0390fd8002a0fb1b56"} err="failed to get container status \"084259d551af727f34a854cc0781aea3747ab2650652cf0390fd8002a0fb1b56\": rpc error: code = NotFound desc = could not find container \"084259d551af727f34a854cc0781aea3747ab2650652cf0390fd8002a0fb1b56\": container with ID starting with 084259d551af727f34a854cc0781aea3747ab2650652cf0390fd8002a0fb1b56 not found: ID does not exist"
Jul 07 11:50:04.127288 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:04.127254    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba5bfdf6-54a5-4087-b882-9fd51391cb90-tls-certs\") pod \"ba5bfdf6-54a5-4087-b882-9fd51391cb90\" (UID: \"ba5bfdf6-54a5-4087-b882-9fd51391cb90\") "
Jul 07 11:50:04.127494 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:04.127311    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba5bfdf6-54a5-4087-b882-9fd51391cb90-home\") pod \"ba5bfdf6-54a5-4087-b882-9fd51391cb90\" (UID: \"ba5bfdf6-54a5-4087-b882-9fd51391cb90\") "
Jul 07 11:50:04.127494 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:04.127360    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba5bfdf6-54a5-4087-b882-9fd51391cb90-tmp-dir\") pod \"ba5bfdf6-54a5-4087-b882-9fd51391cb90\" (UID: \"ba5bfdf6-54a5-4087-b882-9fd51391cb90\") "
Jul 07 11:50:04.127494 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:04.127424    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba5bfdf6-54a5-4087-b882-9fd51391cb90-dshm\") pod \"ba5bfdf6-54a5-4087-b882-9fd51391cb90\" (UID: \"ba5bfdf6-54a5-4087-b882-9fd51391cb90\") "
Jul 07 11:50:04.127494 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:04.127457    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba5bfdf6-54a5-4087-b882-9fd51391cb90-model-cache\") pod \"ba5bfdf6-54a5-4087-b882-9fd51391cb90\" (UID: \"ba5bfdf6-54a5-4087-b882-9fd51391cb90\") "
Jul 07 11:50:04.127494 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:04.127488    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9z24z\" (UniqueName: \"kubernetes.io/projected/ba5bfdf6-54a5-4087-b882-9fd51391cb90-kube-api-access-9z24z\") pod \"ba5bfdf6-54a5-4087-b882-9fd51391cb90\" (UID: \"ba5bfdf6-54a5-4087-b882-9fd51391cb90\") "
Jul 07 11:50:04.127783 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:04.127613    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba5bfdf6-54a5-4087-b882-9fd51391cb90-home" (OuterVolumeSpecName: "home") pod "ba5bfdf6-54a5-4087-b882-9fd51391cb90" (UID: "ba5bfdf6-54a5-4087-b882-9fd51391cb90"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:50:04.127783 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:04.127640    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba5bfdf6-54a5-4087-b882-9fd51391cb90-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ba5bfdf6-54a5-4087-b882-9fd51391cb90" (UID: "ba5bfdf6-54a5-4087-b882-9fd51391cb90"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:50:04.127783 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:04.127741    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba5bfdf6-54a5-4087-b882-9fd51391cb90-model-cache" (OuterVolumeSpecName: "model-cache") pod "ba5bfdf6-54a5-4087-b882-9fd51391cb90" (UID: "ba5bfdf6-54a5-4087-b882-9fd51391cb90"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:50:04.127966 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:04.127877    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba5bfdf6-54a5-4087-b882-9fd51391cb90-model-cache\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:50:04.127966 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:04.127895    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba5bfdf6-54a5-4087-b882-9fd51391cb90-home\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:50:04.127966 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:04.127908    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba5bfdf6-54a5-4087-b882-9fd51391cb90-tmp-dir\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:50:04.129754 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:04.129725    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba5bfdf6-54a5-4087-b882-9fd51391cb90-dshm" (OuterVolumeSpecName: "dshm") pod "ba5bfdf6-54a5-4087-b882-9fd51391cb90" (UID: "ba5bfdf6-54a5-4087-b882-9fd51391cb90"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:50:04.129845 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:04.129781    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ba5bfdf6-54a5-4087-b882-9fd51391cb90-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ba5bfdf6-54a5-4087-b882-9fd51391cb90" (UID: "ba5bfdf6-54a5-4087-b882-9fd51391cb90"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 11:50:04.129886 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:04.129874    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba5bfdf6-54a5-4087-b882-9fd51391cb90-kube-api-access-9z24z" (OuterVolumeSpecName: "kube-api-access-9z24z") pod "ba5bfdf6-54a5-4087-b882-9fd51391cb90" (UID: "ba5bfdf6-54a5-4087-b882-9fd51391cb90"). InnerVolumeSpecName "kube-api-access-9z24z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:50:04.228998 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:04.228960    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9z24z\" (UniqueName: \"kubernetes.io/projected/ba5bfdf6-54a5-4087-b882-9fd51391cb90-kube-api-access-9z24z\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:50:04.228998 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:04.228995    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba5bfdf6-54a5-4087-b882-9fd51391cb90-tls-certs\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:50:04.228998 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:04.229007    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba5bfdf6-54a5-4087-b882-9fd51391cb90-dshm\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:50:04.414660 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:04.414616    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b5d0aa/sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp"]
Jul 07 11:50:04.420871 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:04.420838    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-b5d0aa/sub-test-model-b5d0aa-kserve-7d45cb5597-j8jhp"]
Jul 07 11:50:04.822262 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:04.822230    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba5bfdf6-54a5-4087-b882-9fd51391cb90" path="/var/lib/kubelet/pods/ba5bfdf6-54a5-4087-b882-9fd51391cb90/volumes"
Jul 07 11:50:15.729953 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:15.729916    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-43ebe2/rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q"]
Jul 07 11:50:15.730491 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:15.730306    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba5bfdf6-54a5-4087-b882-9fd51391cb90" containerName="main"
Jul 07 11:50:15.730491 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:15.730317    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba5bfdf6-54a5-4087-b882-9fd51391cb90" containerName="main"
Jul 07 11:50:15.730491 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:15.730403    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba5bfdf6-54a5-4087-b882-9fd51391cb90" containerName="main"
Jul 07 11:50:15.734750 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:15.734729    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-43ebe2/rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q"
Jul 07 11:50:15.737996 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:15.737974    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-43ebe2\"/\"openshift-service-ca.crt\""
Jul 07 11:50:15.738132 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:15.737983    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-43ebe2\"/\"kube-root-ca.crt\""
Jul 07 11:50:15.739397 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:15.739378    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-43ebe2\"/\"default-dockercfg-8dc6c\""
Jul 07 11:50:15.739397 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:15.739391    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-43ebe2\"/\"rate-test-model-43ebe2-kserve-self-signed-certs\""
Jul 07 11:50:15.744043 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:15.744021    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-43ebe2/rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q"]
Jul 07 11:50:15.840319 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:15.840276    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58cfaabd-a2a7-4310-ae41-3d2e328ce664-home\") pod \"rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q\" (UID: \"58cfaabd-a2a7-4310-ae41-3d2e328ce664\") " pod="ai-tenant-e2e-shared-b-43ebe2/rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q"
Jul 07 11:50:15.840585 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:15.840365    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58cfaabd-a2a7-4310-ae41-3d2e328ce664-tmp-dir\") pod \"rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q\" (UID: \"58cfaabd-a2a7-4310-ae41-3d2e328ce664\") " pod="ai-tenant-e2e-shared-b-43ebe2/rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q"
Jul 07 11:50:15.840585 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:15.840389    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmjqp\" (UniqueName: \"kubernetes.io/projected/58cfaabd-a2a7-4310-ae41-3d2e328ce664-kube-api-access-pmjqp\") pod \"rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q\" (UID: \"58cfaabd-a2a7-4310-ae41-3d2e328ce664\") " pod="ai-tenant-e2e-shared-b-43ebe2/rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q"
Jul 07 11:50:15.840585 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:15.840409    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58cfaabd-a2a7-4310-ae41-3d2e328ce664-model-cache\") pod \"rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q\" (UID: \"58cfaabd-a2a7-4310-ae41-3d2e328ce664\") " pod="ai-tenant-e2e-shared-b-43ebe2/rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q"
Jul 07 11:50:15.840585 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:15.840447    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58cfaabd-a2a7-4310-ae41-3d2e328ce664-tls-certs\") pod \"rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q\" (UID: \"58cfaabd-a2a7-4310-ae41-3d2e328ce664\") " pod="ai-tenant-e2e-shared-b-43ebe2/rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q"
Jul 07 11:50:15.840585 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:15.840531    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58cfaabd-a2a7-4310-ae41-3d2e328ce664-dshm\") pod \"rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q\" (UID: \"58cfaabd-a2a7-4310-ae41-3d2e328ce664\") " pod="ai-tenant-e2e-shared-b-43ebe2/rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q"
Jul 07 11:50:15.941728 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:15.941688    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58cfaabd-a2a7-4310-ae41-3d2e328ce664-tls-certs\") pod \"rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q\" (UID: \"58cfaabd-a2a7-4310-ae41-3d2e328ce664\") " pod="ai-tenant-e2e-shared-b-43ebe2/rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q"
Jul 07 11:50:15.941728 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:15.941729    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58cfaabd-a2a7-4310-ae41-3d2e328ce664-dshm\") pod \"rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q\" (UID: \"58cfaabd-a2a7-4310-ae41-3d2e328ce664\") " pod="ai-tenant-e2e-shared-b-43ebe2/rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q"
Jul 07 11:50:15.941949 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:15.941799    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58cfaabd-a2a7-4310-ae41-3d2e328ce664-home\") pod \"rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q\" (UID: \"58cfaabd-a2a7-4310-ae41-3d2e328ce664\") " pod="ai-tenant-e2e-shared-b-43ebe2/rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q"
Jul 07 11:50:15.941949 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:15.941829    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58cfaabd-a2a7-4310-ae41-3d2e328ce664-tmp-dir\") pod \"rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q\" (UID: \"58cfaabd-a2a7-4310-ae41-3d2e328ce664\") " pod="ai-tenant-e2e-shared-b-43ebe2/rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q"
Jul 07 11:50:15.941949 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:15.941853    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmjqp\" (UniqueName: \"kubernetes.io/projected/58cfaabd-a2a7-4310-ae41-3d2e328ce664-kube-api-access-pmjqp\") pod \"rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q\" (UID: \"58cfaabd-a2a7-4310-ae41-3d2e328ce664\") " pod="ai-tenant-e2e-shared-b-43ebe2/rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q"
Jul 07 11:50:15.942088 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:15.941985    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58cfaabd-a2a7-4310-ae41-3d2e328ce664-model-cache\") pod \"rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q\" (UID: \"58cfaabd-a2a7-4310-ae41-3d2e328ce664\") " pod="ai-tenant-e2e-shared-b-43ebe2/rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q"
Jul 07 11:50:15.942274 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:15.942252    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58cfaabd-a2a7-4310-ae41-3d2e328ce664-tmp-dir\") pod \"rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q\" (UID: \"58cfaabd-a2a7-4310-ae41-3d2e328ce664\") " pod="ai-tenant-e2e-shared-b-43ebe2/rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q"
Jul 07 11:50:15.942411 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:15.942308    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58cfaabd-a2a7-4310-ae41-3d2e328ce664-home\") pod \"rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q\" (UID: \"58cfaabd-a2a7-4310-ae41-3d2e328ce664\") " pod="ai-tenant-e2e-shared-b-43ebe2/rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q"
Jul 07 11:50:15.942411 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:15.942373    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58cfaabd-a2a7-4310-ae41-3d2e328ce664-model-cache\") pod \"rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q\" (UID: \"58cfaabd-a2a7-4310-ae41-3d2e328ce664\") " pod="ai-tenant-e2e-shared-b-43ebe2/rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q"
Jul 07 11:50:15.944275 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:15.944245    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58cfaabd-a2a7-4310-ae41-3d2e328ce664-dshm\") pod \"rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q\" (UID: \"58cfaabd-a2a7-4310-ae41-3d2e328ce664\") " pod="ai-tenant-e2e-shared-b-43ebe2/rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q"
Jul 07 11:50:15.944463 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:15.944448    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58cfaabd-a2a7-4310-ae41-3d2e328ce664-tls-certs\") pod \"rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q\" (UID: \"58cfaabd-a2a7-4310-ae41-3d2e328ce664\") " pod="ai-tenant-e2e-shared-b-43ebe2/rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q"
Jul 07 11:50:15.950300 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:15.950276    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmjqp\" (UniqueName: \"kubernetes.io/projected/58cfaabd-a2a7-4310-ae41-3d2e328ce664-kube-api-access-pmjqp\") pod \"rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q\" (UID: \"58cfaabd-a2a7-4310-ae41-3d2e328ce664\") " pod="ai-tenant-e2e-shared-b-43ebe2/rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q"
Jul 07 11:50:16.044737 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:16.044631    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-43ebe2/rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q"
Jul 07 11:50:16.193237 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:16.193210    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-43ebe2/rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q"]
Jul 07 11:50:16.195386 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:50:16.195354    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod58cfaabd_a2a7_4310_ae41_3d2e328ce664.slice/crio-745ce3a9ce943d96d84a05ff31035f82b11efeafcc8a5c9511c2913451442eba WatchSource:0}: Error finding container 745ce3a9ce943d96d84a05ff31035f82b11efeafcc8a5c9511c2913451442eba: Status 404 returned error can't find the container with id 745ce3a9ce943d96d84a05ff31035f82b11efeafcc8a5c9511c2913451442eba
Jul 07 11:50:17.153190 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:17.153159    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-43ebe2/rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q" event={"ID":"58cfaabd-a2a7-4310-ae41-3d2e328ce664","Type":"ContainerStarted","Data":"9e2d41b47e01c57cb082292b2e5e798ef128dc8c24e4c89de5f4ae87e536bad5"}
Jul 07 11:50:17.153621 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:17.153199    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-43ebe2/rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q" event={"ID":"58cfaabd-a2a7-4310-ae41-3d2e328ce664","Type":"ContainerStarted","Data":"745ce3a9ce943d96d84a05ff31035f82b11efeafcc8a5c9511c2913451442eba"}
Jul 07 11:50:17.180487 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:17.180434    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-43ebe2/rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q" podStartSLOduration=2.180419335 podStartE2EDuration="2.180419335s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:50:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:50:17.177486694 +0000 UTC m=+2366.887383091" watchObservedRunningTime="2026-07-07 11:50:17.180419335 +0000 UTC m=+2366.890315722"
Jul 07 11:50:26.044870 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:26.044815    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-43ebe2/rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q"
Jul 07 11:50:26.044870 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:26.044880    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-43ebe2/rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q"
Jul 07 11:50:26.057388 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:26.057364    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-43ebe2/rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q"
Jul 07 11:50:26.210168 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:26.210138    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-43ebe2/rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q"
Jul 07 11:50:50.924237 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:50.924201    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4v7v8_c8c17c90-3e07-4530-8411-d59cef9a1aff/console-operator/2.log"
Jul 07 11:50:50.927574 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:50:50.927554    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4v7v8_c8c17c90-3e07-4530-8411-d59cef9a1aff/console-operator/2.log"
Jul 07 11:51:08.912666 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:08.912629    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b5d0aa/rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7"]
Jul 07 11:51:09.792031 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:09.791997    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-43ebe2/rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q"]
Jul 07 11:51:23.933904 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:23.933836    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-b5d0aa/rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7" podUID="978a6abb-16d0-41a9-b4cd-1f16e8ff7a67" containerName="main" containerID="cri-o://a3d37cfcc6f0a5112edff9f945dfa37c913a06118368f37240ed31347ce6f303" gracePeriod=45
Jul 07 11:51:24.192737 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:24.192675    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b5d0aa/rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7"
Jul 07 11:51:24.265585 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:24.265553    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6rq5x\" (UniqueName: \"kubernetes.io/projected/978a6abb-16d0-41a9-b4cd-1f16e8ff7a67-kube-api-access-6rq5x\") pod \"978a6abb-16d0-41a9-b4cd-1f16e8ff7a67\" (UID: \"978a6abb-16d0-41a9-b4cd-1f16e8ff7a67\") "
Jul 07 11:51:24.265753 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:24.265594    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/978a6abb-16d0-41a9-b4cd-1f16e8ff7a67-home\") pod \"978a6abb-16d0-41a9-b4cd-1f16e8ff7a67\" (UID: \"978a6abb-16d0-41a9-b4cd-1f16e8ff7a67\") "
Jul 07 11:51:24.265753 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:24.265675    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/978a6abb-16d0-41a9-b4cd-1f16e8ff7a67-dshm\") pod \"978a6abb-16d0-41a9-b4cd-1f16e8ff7a67\" (UID: \"978a6abb-16d0-41a9-b4cd-1f16e8ff7a67\") "
Jul 07 11:51:24.265753 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:24.265698    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/978a6abb-16d0-41a9-b4cd-1f16e8ff7a67-tls-certs\") pod \"978a6abb-16d0-41a9-b4cd-1f16e8ff7a67\" (UID: \"978a6abb-16d0-41a9-b4cd-1f16e8ff7a67\") "
Jul 07 11:51:24.265753 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:24.265736    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/978a6abb-16d0-41a9-b4cd-1f16e8ff7a67-tmp-dir\") pod \"978a6abb-16d0-41a9-b4cd-1f16e8ff7a67\" (UID: \"978a6abb-16d0-41a9-b4cd-1f16e8ff7a67\") "
Jul 07 11:51:24.265974 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:24.265755    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/978a6abb-16d0-41a9-b4cd-1f16e8ff7a67-model-cache\") pod \"978a6abb-16d0-41a9-b4cd-1f16e8ff7a67\" (UID: \"978a6abb-16d0-41a9-b4cd-1f16e8ff7a67\") "
Jul 07 11:51:24.265974 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:24.265922    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/978a6abb-16d0-41a9-b4cd-1f16e8ff7a67-home" (OuterVolumeSpecName: "home") pod "978a6abb-16d0-41a9-b4cd-1f16e8ff7a67" (UID: "978a6abb-16d0-41a9-b4cd-1f16e8ff7a67"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:51:24.266105 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:24.266071    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/978a6abb-16d0-41a9-b4cd-1f16e8ff7a67-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "978a6abb-16d0-41a9-b4cd-1f16e8ff7a67" (UID: "978a6abb-16d0-41a9-b4cd-1f16e8ff7a67"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:51:24.266105 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:24.266100    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/978a6abb-16d0-41a9-b4cd-1f16e8ff7a67-home\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:51:24.266191 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:24.266105    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/978a6abb-16d0-41a9-b4cd-1f16e8ff7a67-model-cache" (OuterVolumeSpecName: "model-cache") pod "978a6abb-16d0-41a9-b4cd-1f16e8ff7a67" (UID: "978a6abb-16d0-41a9-b4cd-1f16e8ff7a67"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:51:24.267967 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:24.267938    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/978a6abb-16d0-41a9-b4cd-1f16e8ff7a67-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "978a6abb-16d0-41a9-b4cd-1f16e8ff7a67" (UID: "978a6abb-16d0-41a9-b4cd-1f16e8ff7a67"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 11:51:24.268067 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:24.267978    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/978a6abb-16d0-41a9-b4cd-1f16e8ff7a67-dshm" (OuterVolumeSpecName: "dshm") pod "978a6abb-16d0-41a9-b4cd-1f16e8ff7a67" (UID: "978a6abb-16d0-41a9-b4cd-1f16e8ff7a67"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:51:24.268067 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:24.267992    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/978a6abb-16d0-41a9-b4cd-1f16e8ff7a67-kube-api-access-6rq5x" (OuterVolumeSpecName: "kube-api-access-6rq5x") pod "978a6abb-16d0-41a9-b4cd-1f16e8ff7a67" (UID: "978a6abb-16d0-41a9-b4cd-1f16e8ff7a67"). InnerVolumeSpecName "kube-api-access-6rq5x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:51:24.367539 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:24.367502    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/978a6abb-16d0-41a9-b4cd-1f16e8ff7a67-dshm\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:51:24.367539 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:24.367532    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/978a6abb-16d0-41a9-b4cd-1f16e8ff7a67-tls-certs\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:51:24.367539 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:24.367542    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/978a6abb-16d0-41a9-b4cd-1f16e8ff7a67-tmp-dir\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:51:24.367539 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:24.367549    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/978a6abb-16d0-41a9-b4cd-1f16e8ff7a67-model-cache\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:51:24.367810 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:24.367558    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6rq5x\" (UniqueName: \"kubernetes.io/projected/978a6abb-16d0-41a9-b4cd-1f16e8ff7a67-kube-api-access-6rq5x\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:51:24.485969 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:24.485932    2570 generic.go:356] "Generic (PLEG): container finished" podID="978a6abb-16d0-41a9-b4cd-1f16e8ff7a67" containerID="a3d37cfcc6f0a5112edff9f945dfa37c913a06118368f37240ed31347ce6f303" exitCode=0
Jul 07 11:51:24.486129 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:24.485979    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b5d0aa/rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7"
Jul 07 11:51:24.486129 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:24.486009    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b5d0aa/rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7" event={"ID":"978a6abb-16d0-41a9-b4cd-1f16e8ff7a67","Type":"ContainerDied","Data":"a3d37cfcc6f0a5112edff9f945dfa37c913a06118368f37240ed31347ce6f303"}
Jul 07 11:51:24.486129 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:24.486052    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b5d0aa/rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7" event={"ID":"978a6abb-16d0-41a9-b4cd-1f16e8ff7a67","Type":"ContainerDied","Data":"a7590eee7fba867edf27510eaee66f466212114fe369dd2592a3b5f2a609ab46"}
Jul 07 11:51:24.486129 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:24.486069    2570 scope.go:117] "RemoveContainer" containerID="a3d37cfcc6f0a5112edff9f945dfa37c913a06118368f37240ed31347ce6f303"
Jul 07 11:51:24.495573 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:24.495560    2570 scope.go:117] "RemoveContainer" containerID="a3d37cfcc6f0a5112edff9f945dfa37c913a06118368f37240ed31347ce6f303"
Jul 07 11:51:24.495806 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:51:24.495778    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3d37cfcc6f0a5112edff9f945dfa37c913a06118368f37240ed31347ce6f303\": container with ID starting with a3d37cfcc6f0a5112edff9f945dfa37c913a06118368f37240ed31347ce6f303 not found: ID does not exist" containerID="a3d37cfcc6f0a5112edff9f945dfa37c913a06118368f37240ed31347ce6f303"
Jul 07 11:51:24.495854 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:24.495816    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3d37cfcc6f0a5112edff9f945dfa37c913a06118368f37240ed31347ce6f303"} err="failed to get container status \"a3d37cfcc6f0a5112edff9f945dfa37c913a06118368f37240ed31347ce6f303\": rpc error: code = NotFound desc = could not find container \"a3d37cfcc6f0a5112edff9f945dfa37c913a06118368f37240ed31347ce6f303\": container with ID starting with a3d37cfcc6f0a5112edff9f945dfa37c913a06118368f37240ed31347ce6f303 not found: ID does not exist"
Jul 07 11:51:24.508175 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:24.508150    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b5d0aa/rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7"]
Jul 07 11:51:24.512606 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:24.512584    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-b5d0aa/rate-test-model-b5d0aa-kserve-86b78fb44c-d9lw7"]
Jul 07 11:51:24.812502 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:24.812402    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-43ebe2/rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q" podUID="58cfaabd-a2a7-4310-ae41-3d2e328ce664" containerName="main" containerID="cri-o://9e2d41b47e01c57cb082292b2e5e798ef128dc8c24e4c89de5f4ae87e536bad5" gracePeriod=45
Jul 07 11:51:24.823096 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:24.823064    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="978a6abb-16d0-41a9-b4cd-1f16e8ff7a67" path="/var/lib/kubelet/pods/978a6abb-16d0-41a9-b4cd-1f16e8ff7a67/volumes"
Jul 07 11:51:25.071691 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:25.071624    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-43ebe2/rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q"
Jul 07 11:51:25.176427 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:25.176398    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58cfaabd-a2a7-4310-ae41-3d2e328ce664-home\") pod \"58cfaabd-a2a7-4310-ae41-3d2e328ce664\" (UID: \"58cfaabd-a2a7-4310-ae41-3d2e328ce664\") "
Jul 07 11:51:25.176427 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:25.176433    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pmjqp\" (UniqueName: \"kubernetes.io/projected/58cfaabd-a2a7-4310-ae41-3d2e328ce664-kube-api-access-pmjqp\") pod \"58cfaabd-a2a7-4310-ae41-3d2e328ce664\" (UID: \"58cfaabd-a2a7-4310-ae41-3d2e328ce664\") "
Jul 07 11:51:25.176673 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:25.176469    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58cfaabd-a2a7-4310-ae41-3d2e328ce664-tls-certs\") pod \"58cfaabd-a2a7-4310-ae41-3d2e328ce664\" (UID: \"58cfaabd-a2a7-4310-ae41-3d2e328ce664\") "
Jul 07 11:51:25.176673 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:25.176488    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58cfaabd-a2a7-4310-ae41-3d2e328ce664-model-cache\") pod \"58cfaabd-a2a7-4310-ae41-3d2e328ce664\" (UID: \"58cfaabd-a2a7-4310-ae41-3d2e328ce664\") "
Jul 07 11:51:25.176673 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:25.176561    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58cfaabd-a2a7-4310-ae41-3d2e328ce664-dshm\") pod \"58cfaabd-a2a7-4310-ae41-3d2e328ce664\" (UID: \"58cfaabd-a2a7-4310-ae41-3d2e328ce664\") "
Jul 07 11:51:25.176673 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:25.176594    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58cfaabd-a2a7-4310-ae41-3d2e328ce664-tmp-dir\") pod \"58cfaabd-a2a7-4310-ae41-3d2e328ce664\" (UID: \"58cfaabd-a2a7-4310-ae41-3d2e328ce664\") "
Jul 07 11:51:25.176936 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:25.176729    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58cfaabd-a2a7-4310-ae41-3d2e328ce664-home" (OuterVolumeSpecName: "home") pod "58cfaabd-a2a7-4310-ae41-3d2e328ce664" (UID: "58cfaabd-a2a7-4310-ae41-3d2e328ce664"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:51:25.176936 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:25.176792    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58cfaabd-a2a7-4310-ae41-3d2e328ce664-model-cache" (OuterVolumeSpecName: "model-cache") pod "58cfaabd-a2a7-4310-ae41-3d2e328ce664" (UID: "58cfaabd-a2a7-4310-ae41-3d2e328ce664"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:51:25.176936 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:25.176921    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58cfaabd-a2a7-4310-ae41-3d2e328ce664-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "58cfaabd-a2a7-4310-ae41-3d2e328ce664" (UID: "58cfaabd-a2a7-4310-ae41-3d2e328ce664"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:51:25.177040 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:25.176956    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58cfaabd-a2a7-4310-ae41-3d2e328ce664-model-cache\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:51:25.177040 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:25.176968    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58cfaabd-a2a7-4310-ae41-3d2e328ce664-home\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:51:25.178755 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:25.178734    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58cfaabd-a2a7-4310-ae41-3d2e328ce664-dshm" (OuterVolumeSpecName: "dshm") pod "58cfaabd-a2a7-4310-ae41-3d2e328ce664" (UID: "58cfaabd-a2a7-4310-ae41-3d2e328ce664"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:51:25.179353 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:25.179314    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58cfaabd-a2a7-4310-ae41-3d2e328ce664-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "58cfaabd-a2a7-4310-ae41-3d2e328ce664" (UID: "58cfaabd-a2a7-4310-ae41-3d2e328ce664"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 11:51:25.179434 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:25.179353    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58cfaabd-a2a7-4310-ae41-3d2e328ce664-kube-api-access-pmjqp" (OuterVolumeSpecName: "kube-api-access-pmjqp") pod "58cfaabd-a2a7-4310-ae41-3d2e328ce664" (UID: "58cfaabd-a2a7-4310-ae41-3d2e328ce664"). InnerVolumeSpecName "kube-api-access-pmjqp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:51:25.278253 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:25.278210    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58cfaabd-a2a7-4310-ae41-3d2e328ce664-tmp-dir\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:51:25.278253 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:25.278247    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pmjqp\" (UniqueName: \"kubernetes.io/projected/58cfaabd-a2a7-4310-ae41-3d2e328ce664-kube-api-access-pmjqp\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:51:25.278253 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:25.278261    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58cfaabd-a2a7-4310-ae41-3d2e328ce664-tls-certs\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:51:25.278538 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:25.278273    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58cfaabd-a2a7-4310-ae41-3d2e328ce664-dshm\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:51:25.493095 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:25.493065    2570 generic.go:356] "Generic (PLEG): container finished" podID="58cfaabd-a2a7-4310-ae41-3d2e328ce664" containerID="9e2d41b47e01c57cb082292b2e5e798ef128dc8c24e4c89de5f4ae87e536bad5" exitCode=0
Jul 07 11:51:25.493262 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:25.493102    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-43ebe2/rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q"
Jul 07 11:51:25.493262 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:25.493110    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-43ebe2/rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q" event={"ID":"58cfaabd-a2a7-4310-ae41-3d2e328ce664","Type":"ContainerDied","Data":"9e2d41b47e01c57cb082292b2e5e798ef128dc8c24e4c89de5f4ae87e536bad5"}
Jul 07 11:51:25.493262 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:25.493149    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-43ebe2/rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q" event={"ID":"58cfaabd-a2a7-4310-ae41-3d2e328ce664","Type":"ContainerDied","Data":"745ce3a9ce943d96d84a05ff31035f82b11efeafcc8a5c9511c2913451442eba"}
Jul 07 11:51:25.493262 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:25.493169    2570 scope.go:117] "RemoveContainer" containerID="9e2d41b47e01c57cb082292b2e5e798ef128dc8c24e4c89de5f4ae87e536bad5"
Jul 07 11:51:25.502037 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:25.502020    2570 scope.go:117] "RemoveContainer" containerID="9e2d41b47e01c57cb082292b2e5e798ef128dc8c24e4c89de5f4ae87e536bad5"
Jul 07 11:51:25.502294 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:51:25.502273    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e2d41b47e01c57cb082292b2e5e798ef128dc8c24e4c89de5f4ae87e536bad5\": container with ID starting with 9e2d41b47e01c57cb082292b2e5e798ef128dc8c24e4c89de5f4ae87e536bad5 not found: ID does not exist" containerID="9e2d41b47e01c57cb082292b2e5e798ef128dc8c24e4c89de5f4ae87e536bad5"
Jul 07 11:51:25.502357 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:25.502303    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e2d41b47e01c57cb082292b2e5e798ef128dc8c24e4c89de5f4ae87e536bad5"} err="failed to get container status \"9e2d41b47e01c57cb082292b2e5e798ef128dc8c24e4c89de5f4ae87e536bad5\": rpc error: code = NotFound desc = could not find container \"9e2d41b47e01c57cb082292b2e5e798ef128dc8c24e4c89de5f4ae87e536bad5\": container with ID starting with 9e2d41b47e01c57cb082292b2e5e798ef128dc8c24e4c89de5f4ae87e536bad5 not found: ID does not exist"
Jul 07 11:51:25.518909 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:25.518883    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-43ebe2/rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q"]
Jul 07 11:51:25.523348 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:25.523316    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-43ebe2/rate-test-model-43ebe2-kserve-55475c7bb7-l2q5q"]
Jul 07 11:51:26.821607 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:51:26.821574    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58cfaabd-a2a7-4310-ae41-3d2e328ce664" path="/var/lib/kubelet/pods/58cfaabd-a2a7-4310-ae41-3d2e328ce664/volumes"
Jul 07 11:52:19.369316 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:52:19.369283    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-b5d0aa-65cbc9b5f5-g4bkh"]
Jul 07 11:52:19.371588 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:52:19.369518    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-a-b5d0aa-65cbc9b5f5-g4bkh" podUID="2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e" containerName="maas-api" containerID="cri-o://fcbba72f6b249765559376e13d815ee242376f4b3d9d559b7b0154ab1c2123b9" gracePeriod=30
Jul 07 11:52:19.622816 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:52:19.622753    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-b5d0aa-65cbc9b5f5-g4bkh"
Jul 07 11:52:19.673287 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:52:19.673249    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e-maas-api-tls\") pod \"2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e\" (UID: \"2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e\") "
Jul 07 11:52:19.673497 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:52:19.673365    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xcn86\" (UniqueName: \"kubernetes.io/projected/2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e-kube-api-access-xcn86\") pod \"2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e\" (UID: \"2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e\") "
Jul 07 11:52:19.673497 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:52:19.673413    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e-openshift-service-ca\") pod \"2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e\" (UID: \"2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e\") "
Jul 07 11:52:19.673757 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:52:19.673729    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e" (UID: "2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 11:52:19.675614 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:52:19.675584    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e" (UID: "2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 11:52:19.675726 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:52:19.675634    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e-kube-api-access-xcn86" (OuterVolumeSpecName: "kube-api-access-xcn86") pod "2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e" (UID: "2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e"). InnerVolumeSpecName "kube-api-access-xcn86". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:52:19.763497 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:52:19.763465    2570 generic.go:356] "Generic (PLEG): container finished" podID="2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e" containerID="fcbba72f6b249765559376e13d815ee242376f4b3d9d559b7b0154ab1c2123b9" exitCode=0
Jul 07 11:52:19.763685 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:52:19.763511    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-b5d0aa-65cbc9b5f5-g4bkh"
Jul 07 11:52:19.763685 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:52:19.763514    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-b5d0aa-65cbc9b5f5-g4bkh" event={"ID":"2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e","Type":"ContainerDied","Data":"fcbba72f6b249765559376e13d815ee242376f4b3d9d559b7b0154ab1c2123b9"}
Jul 07 11:52:19.763685 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:52:19.763571    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-b5d0aa-65cbc9b5f5-g4bkh" event={"ID":"2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e","Type":"ContainerDied","Data":"cafe44ff06806cdfd0e2e85a45e151a5b5f965af54f101b553b7b1ec4f42f254"}
Jul 07 11:52:19.763685 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:52:19.763609    2570 scope.go:117] "RemoveContainer" containerID="fcbba72f6b249765559376e13d815ee242376f4b3d9d559b7b0154ab1c2123b9"
Jul 07 11:52:19.773439 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:52:19.773416    2570 scope.go:117] "RemoveContainer" containerID="fcbba72f6b249765559376e13d815ee242376f4b3d9d559b7b0154ab1c2123b9"
Jul 07 11:52:19.773731 ip-10-0-142-39 kubenswrapper[2570]: E0707 11:52:19.773712    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fcbba72f6b249765559376e13d815ee242376f4b3d9d559b7b0154ab1c2123b9\": container with ID starting with fcbba72f6b249765559376e13d815ee242376f4b3d9d559b7b0154ab1c2123b9 not found: ID does not exist" containerID="fcbba72f6b249765559376e13d815ee242376f4b3d9d559b7b0154ab1c2123b9"
Jul 07 11:52:19.773809 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:52:19.773745    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcbba72f6b249765559376e13d815ee242376f4b3d9d559b7b0154ab1c2123b9"} err="failed to get container status \"fcbba72f6b249765559376e13d815ee242376f4b3d9d559b7b0154ab1c2123b9\": rpc error: code = NotFound desc = could not find container \"fcbba72f6b249765559376e13d815ee242376f4b3d9d559b7b0154ab1c2123b9\": container with ID starting with fcbba72f6b249765559376e13d815ee242376f4b3d9d559b7b0154ab1c2123b9 not found: ID does not exist"
Jul 07 11:52:19.774038 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:52:19.774023    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xcn86\" (UniqueName: \"kubernetes.io/projected/2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e-kube-api-access-xcn86\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:52:19.774075 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:52:19.774042    2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e-openshift-service-ca\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:52:19.774075 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:52:19.774052    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e-maas-api-tls\") on node \"ip-10-0-142-39.ec2.internal\" DevicePath \"\""
Jul 07 11:52:19.787446 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:52:19.787416    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-b5d0aa-65cbc9b5f5-g4bkh"]
Jul 07 11:52:19.792134 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:52:19.792109    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-b5d0aa-65cbc9b5f5-g4bkh"]
Jul 07 11:52:20.822565 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:52:20.822529    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e" path="/var/lib/kubelet/pods/2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e/volumes"
Jul 07 11:52:34.286865 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:52:34.286836    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-55655b5696-4nnbg_b81abf77-b887-4ef6-9b9d-b6c1f94277db/authorino/0.log"
Jul 07 11:52:39.018982 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:52:39.018950    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-9k6t8_c4efd1dc-7a63-47d3-a8af-ce5e8e9402e1/manager/0.log"
Jul 07 11:52:39.138118 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:52:39.138087    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-ht2l6_282ba625-e484-45e0-b00a-962c5431ffba/manager/0.log"
Jul 07 11:52:39.262655 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:52:39.262621    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-6c8bcdc88b-7wwlc_71ef84c7-9a12-42d5-b95b-410fe33993ef/maas-api/0.log"
Jul 07 11:52:39.628012 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:52:39.627974    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7bd7d6c999-df6bp_621d0ca4-4868-4d28-b7db-51e4a459338c/manager/0.log"
Jul 07 11:52:39.847888 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:52:39.847848    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-qrhf8_85974022-4d26-4e54-bda4-3b39cce716cd/manager/0.log"
Jul 07 11:52:39.962122 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:52:39.962039    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6dc7dd79cd-bxzht_948ae785-e791-40b1-a7c8-029a6e9e3d8e/copy-manifests/0.log"
Jul 07 11:52:39.969725 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:52:39.969697    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6dc7dd79cd-bxzht_948ae785-e791-40b1-a7c8-029a6e9e3d8e/manager/0.log"
Jul 07 11:52:41.740347 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:52:41.740241    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-55655b5696-4nnbg_b81abf77-b887-4ef6-9b9d-b6c1f94277db/authorino/0.log"
Jul 07 11:52:41.861199 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:52:41.861164    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-lmghr_3ea4410a-8584-4082-81a7-44b639de86a7/manager/0.log"
Jul 07 11:52:42.196290 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:52:42.196207    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-lccrg_a9b73f5f-f512-4ffd-a7dc-ba6c945d7414/registry-server/0.log"
Jul 07 11:52:42.542106 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:52:42.542072    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-crrvk_a8053105-3f63-440e-bc40-528c2c56344c/manager/0.log"
Jul 07 11:52:43.465376 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:52:43.465347    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59f49fd4f6-lrg8c_397d2fc4-9780-4e14-960c-15519470ad42/payload-pre-processing/0.log"
Jul 07 11:52:43.584400 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:52:43.584368    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bbcb654f4-96trx_0da2d46b-d50f-44cd-97fa-910881673476/payload-processing/0.log"
Jul 07 11:52:43.701058 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:52:43.701027    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5dc4cf6785-bflm8_91203aa4-b720-4390-a2c0-5c553ffd79f8/router/0.log"
Jul 07 11:52:44.259705 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:52:44.259677    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-754fq_0e97b54d-b8dd-40ef-bcf5-4a6ab21e1bbf/main/0.log"
Jul 07 11:52:44.482961 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:52:44.482933    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-jh5w6_d8ba409c-d18b-4602-b567-cc0541953a57/main/0.log"
Jul 07 11:52:44.592973 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:52:44.592896    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-bmr7n_8d99d53d-53bc-4577-8057-70cf8f53f1e7/main/0.log"
Jul 07 11:52:51.609168 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:52:51.609117    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-px74n_a89cad11-0010-4ab5-bc82-04a3cd7a3e7b/global-pull-secret-syncer/0.log"
Jul 07 11:52:51.692961 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:52:51.692913    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-dxztq_c20b78c4-cef9-4f47-ac6e-0ac4303eb857/konnectivity-agent/0.log"
Jul 07 11:52:51.799174 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:52:51.799145    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-39.ec2.internal_7d767ad34fcd7349932e3bffab974a69/haproxy/0.log"
Jul 07 11:52:56.536289 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:52:56.536253    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-55655b5696-4nnbg_b81abf77-b887-4ef6-9b9d-b6c1f94277db/authorino/0.log"
Jul 07 11:52:56.621723 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:52:56.621679    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-lmghr_3ea4410a-8584-4082-81a7-44b639de86a7/manager/0.log"
Jul 07 11:52:56.764745 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:52:56.764714    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-lccrg_a9b73f5f-f512-4ffd-a7dc-ba6c945d7414/registry-server/0.log"
Jul 07 11:52:57.065660 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:52:57.065630    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-crrvk_a8053105-3f63-440e-bc40-528c2c56344c/manager/0.log"
Jul 07 11:52:58.844489 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:52:58.844460    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-9pdtd_df6d16a6-2593-4477-ae7a-eca848817f34/cluster-monitoring-operator/0.log"
Jul 07 11:52:59.376445 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:52:59.376412    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5gjjf_773efbaa-d3a5-4339-96e0-432ce18032ca/node-exporter/0.log"
Jul 07 11:52:59.410672 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:52:59.410644    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5gjjf_773efbaa-d3a5-4339-96e0-432ce18032ca/kube-rbac-proxy/0.log"
Jul 07 11:52:59.450353 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:52:59.450300    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5gjjf_773efbaa-d3a5-4339-96e0-432ce18032ca/init-textfile/0.log"
Jul 07 11:53:00.856997 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:00.856965    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rl7lp/perf-node-gather-daemonset-vtz7m"]
Jul 07 11:53:00.857524 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:00.857503    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58cfaabd-a2a7-4310-ae41-3d2e328ce664" containerName="main"
Jul 07 11:53:00.857604 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:00.857529    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="58cfaabd-a2a7-4310-ae41-3d2e328ce664" containerName="main"
Jul 07 11:53:00.857604 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:00.857565    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="978a6abb-16d0-41a9-b4cd-1f16e8ff7a67" containerName="main"
Jul 07 11:53:00.857604 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:00.857574    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="978a6abb-16d0-41a9-b4cd-1f16e8ff7a67" containerName="main"
Jul 07 11:53:00.857604 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:00.857598    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e" containerName="maas-api"
Jul 07 11:53:00.857801 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:00.857607    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e" containerName="maas-api"
Jul 07 11:53:00.857801 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:00.857724    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="58cfaabd-a2a7-4310-ae41-3d2e328ce664" containerName="main"
Jul 07 11:53:00.857801 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:00.857737    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="978a6abb-16d0-41a9-b4cd-1f16e8ff7a67" containerName="main"
Jul 07 11:53:00.857801 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:00.857750    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="2cfc83d1-5d8c-4cf1-a429-bbf08fb5b62e" containerName="maas-api"
Jul 07 11:53:00.860895 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:00.860876    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rl7lp/perf-node-gather-daemonset-vtz7m"
Jul 07 11:53:00.863995 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:00.863976    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rl7lp\"/\"openshift-service-ca.crt\""
Jul 07 11:53:00.865282 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:00.865262    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-rl7lp\"/\"default-dockercfg-2t97q\""
Jul 07 11:53:00.865392 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:00.865286    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rl7lp\"/\"kube-root-ca.crt\""
Jul 07 11:53:00.871913 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:00.871889    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rl7lp/perf-node-gather-daemonset-vtz7m"]
Jul 07 11:53:00.938976 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:00.938944    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c112a222-2eb9-4aa3-8b6d-cbc04336ca4b-sys\") pod \"perf-node-gather-daemonset-vtz7m\" (UID: \"c112a222-2eb9-4aa3-8b6d-cbc04336ca4b\") " pod="openshift-must-gather-rl7lp/perf-node-gather-daemonset-vtz7m"
Jul 07 11:53:00.939162 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:00.939009    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4b8l\" (UniqueName: \"kubernetes.io/projected/c112a222-2eb9-4aa3-8b6d-cbc04336ca4b-kube-api-access-s4b8l\") pod \"perf-node-gather-daemonset-vtz7m\" (UID: \"c112a222-2eb9-4aa3-8b6d-cbc04336ca4b\") " pod="openshift-must-gather-rl7lp/perf-node-gather-daemonset-vtz7m"
Jul 07 11:53:00.939162 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:00.939082    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c112a222-2eb9-4aa3-8b6d-cbc04336ca4b-podres\") pod \"perf-node-gather-daemonset-vtz7m\" (UID: \"c112a222-2eb9-4aa3-8b6d-cbc04336ca4b\") " pod="openshift-must-gather-rl7lp/perf-node-gather-daemonset-vtz7m"
Jul 07 11:53:00.939162 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:00.939121    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c112a222-2eb9-4aa3-8b6d-cbc04336ca4b-proc\") pod \"perf-node-gather-daemonset-vtz7m\" (UID: \"c112a222-2eb9-4aa3-8b6d-cbc04336ca4b\") " pod="openshift-must-gather-rl7lp/perf-node-gather-daemonset-vtz7m"
Jul 07 11:53:00.939162 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:00.939141    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c112a222-2eb9-4aa3-8b6d-cbc04336ca4b-lib-modules\") pod \"perf-node-gather-daemonset-vtz7m\" (UID: \"c112a222-2eb9-4aa3-8b6d-cbc04336ca4b\") " pod="openshift-must-gather-rl7lp/perf-node-gather-daemonset-vtz7m"
Jul 07 11:53:01.040341 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:01.040295    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c112a222-2eb9-4aa3-8b6d-cbc04336ca4b-podres\") pod \"perf-node-gather-daemonset-vtz7m\" (UID: \"c112a222-2eb9-4aa3-8b6d-cbc04336ca4b\") " pod="openshift-must-gather-rl7lp/perf-node-gather-daemonset-vtz7m"
Jul 07 11:53:01.040545 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:01.040367    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c112a222-2eb9-4aa3-8b6d-cbc04336ca4b-proc\") pod \"perf-node-gather-daemonset-vtz7m\" (UID: \"c112a222-2eb9-4aa3-8b6d-cbc04336ca4b\") " pod="openshift-must-gather-rl7lp/perf-node-gather-daemonset-vtz7m"
Jul 07 11:53:01.040545 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:01.040383    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c112a222-2eb9-4aa3-8b6d-cbc04336ca4b-lib-modules\") pod \"perf-node-gather-daemonset-vtz7m\" (UID: \"c112a222-2eb9-4aa3-8b6d-cbc04336ca4b\") " pod="openshift-must-gather-rl7lp/perf-node-gather-daemonset-vtz7m"
Jul 07 11:53:01.040545 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:01.040421    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c112a222-2eb9-4aa3-8b6d-cbc04336ca4b-sys\") pod \"perf-node-gather-daemonset-vtz7m\" (UID: \"c112a222-2eb9-4aa3-8b6d-cbc04336ca4b\") " pod="openshift-must-gather-rl7lp/perf-node-gather-daemonset-vtz7m"
Jul 07 11:53:01.040545 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:01.040472    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4b8l\" (UniqueName: \"kubernetes.io/projected/c112a222-2eb9-4aa3-8b6d-cbc04336ca4b-kube-api-access-s4b8l\") pod \"perf-node-gather-daemonset-vtz7m\" (UID: \"c112a222-2eb9-4aa3-8b6d-cbc04336ca4b\") " pod="openshift-must-gather-rl7lp/perf-node-gather-daemonset-vtz7m"
Jul 07 11:53:01.040545 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:01.040482    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c112a222-2eb9-4aa3-8b6d-cbc04336ca4b-podres\") pod \"perf-node-gather-daemonset-vtz7m\" (UID: \"c112a222-2eb9-4aa3-8b6d-cbc04336ca4b\") " pod="openshift-must-gather-rl7lp/perf-node-gather-daemonset-vtz7m"
Jul 07 11:53:01.040545 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:01.040499    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c112a222-2eb9-4aa3-8b6d-cbc04336ca4b-proc\") pod \"perf-node-gather-daemonset-vtz7m\" (UID: \"c112a222-2eb9-4aa3-8b6d-cbc04336ca4b\") " pod="openshift-must-gather-rl7lp/perf-node-gather-daemonset-vtz7m"
Jul 07 11:53:01.040545 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:01.040539    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c112a222-2eb9-4aa3-8b6d-cbc04336ca4b-sys\") pod \"perf-node-gather-daemonset-vtz7m\" (UID: \"c112a222-2eb9-4aa3-8b6d-cbc04336ca4b\") " pod="openshift-must-gather-rl7lp/perf-node-gather-daemonset-vtz7m"
Jul 07 11:53:01.040792 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:01.040553    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c112a222-2eb9-4aa3-8b6d-cbc04336ca4b-lib-modules\") pod \"perf-node-gather-daemonset-vtz7m\" (UID: \"c112a222-2eb9-4aa3-8b6d-cbc04336ca4b\") " pod="openshift-must-gather-rl7lp/perf-node-gather-daemonset-vtz7m"
Jul 07 11:53:01.051182 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:01.051151    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4b8l\" (UniqueName: \"kubernetes.io/projected/c112a222-2eb9-4aa3-8b6d-cbc04336ca4b-kube-api-access-s4b8l\") pod \"perf-node-gather-daemonset-vtz7m\" (UID: \"c112a222-2eb9-4aa3-8b6d-cbc04336ca4b\") " pod="openshift-must-gather-rl7lp/perf-node-gather-daemonset-vtz7m"
Jul 07 11:53:01.171732 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:01.171646    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rl7lp/perf-node-gather-daemonset-vtz7m"
Jul 07 11:53:01.310663 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:01.310632    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rl7lp/perf-node-gather-daemonset-vtz7m"]
Jul 07 11:53:01.313746 ip-10-0-142-39 kubenswrapper[2570]: W0707 11:53:01.313716    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc112a222_2eb9_4aa3_8b6d_cbc04336ca4b.slice/crio-de073061d83da299f33c0bfa8c1a927583a245254ae63da63b3e1996a93dcb55 WatchSource:0}: Error finding container de073061d83da299f33c0bfa8c1a927583a245254ae63da63b3e1996a93dcb55: Status 404 returned error can't find the container with id de073061d83da299f33c0bfa8c1a927583a245254ae63da63b3e1996a93dcb55
Jul 07 11:53:01.598181 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:01.598147    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-vj4hq_f9ec8255-2877-4c6a-b248-b50d64978f7a/networking-console-plugin/0.log"
Jul 07 11:53:01.956968 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:01.956870    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rl7lp/perf-node-gather-daemonset-vtz7m" event={"ID":"c112a222-2eb9-4aa3-8b6d-cbc04336ca4b","Type":"ContainerStarted","Data":"f67b34eb1e560f4325a8169b1541aced60ce7f04cf6296b80e4bbd4fd25bf84e"}
Jul 07 11:53:01.956968 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:01.956922    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rl7lp/perf-node-gather-daemonset-vtz7m" event={"ID":"c112a222-2eb9-4aa3-8b6d-cbc04336ca4b","Type":"ContainerStarted","Data":"de073061d83da299f33c0bfa8c1a927583a245254ae63da63b3e1996a93dcb55"}
Jul 07 11:53:01.957458 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:01.956979    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-rl7lp/perf-node-gather-daemonset-vtz7m"
Jul 07 11:53:01.976000 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:01.975941    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-rl7lp/perf-node-gather-daemonset-vtz7m" podStartSLOduration=1.975927647 podStartE2EDuration="1.975927647s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:53:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:53:01.975364623 +0000 UTC m=+2531.685261010" watchObservedRunningTime="2026-07-07 11:53:01.975927647 +0000 UTC m=+2531.685824037"
Jul 07 11:53:02.159167 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:02.159137    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4v7v8_c8c17c90-3e07-4530-8411-d59cef9a1aff/console-operator/2.log"
Jul 07 11:53:02.163096 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:02.163072    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-4v7v8_c8c17c90-3e07-4530-8411-d59cef9a1aff/console-operator/3.log"
Jul 07 11:53:03.192695 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:03.192666    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-tt42z_62ea12c1-c37b-4e29-9700-bd826f004e51/volume-data-source-validator/0.log"
Jul 07 11:53:04.295257 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:04.295224    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-64n4d_daf8c8f2-2b41-4800-a056-1f1b1d9714b7/dns/0.log"
Jul 07 11:53:04.318555 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:04.318532    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-64n4d_daf8c8f2-2b41-4800-a056-1f1b1d9714b7/kube-rbac-proxy/0.log"
Jul 07 11:53:04.451272 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:04.451244    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rqrgc_4ea618ab-a367-4d71-b0bd-e82e15e1f080/dns-node-resolver/0.log"
Jul 07 11:53:05.067871 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:05.067840    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-x8c7r_2827544c-22cd-4454-86a0-fc57d09ea878/node-ca/0.log"
Jul 07 11:53:06.488995 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:06.488964    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59f49fd4f6-lrg8c_397d2fc4-9780-4e14-960c-15519470ad42/payload-pre-processing/0.log"
Jul 07 11:53:06.561302 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:06.561273    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bbcb654f4-96trx_0da2d46b-d50f-44cd-97fa-910881673476/payload-processing/0.log"
Jul 07 11:53:06.621312 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:06.621282    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5dc4cf6785-bflm8_91203aa4-b720-4390-a2c0-5c553ffd79f8/router/0.log"
Jul 07 11:53:07.400308 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:07.400282    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wlbnq_3e1e8d03-ef0d-49d9-a760-64039277242c/serve-healthcheck-canary/0.log"
Jul 07 11:53:07.935120 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:07.935088    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-wffz6_6387d3da-1290-437c-8fdc-2422d49dac72/insights-operator/0.log"
Jul 07 11:53:07.935562 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:07.935546    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-wffz6_6387d3da-1290-437c-8fdc-2422d49dac72/insights-operator/1.log"
Jul 07 11:53:07.972298 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:07.972273    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-rl7lp/perf-node-gather-daemonset-vtz7m"
Jul 07 11:53:08.054133 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:08.054108    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8kjb7_cc937ff9-effb-4a24-a888-06fc5f4e5fe6/kube-rbac-proxy/0.log"
Jul 07 11:53:08.080632 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:08.080606    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8kjb7_cc937ff9-effb-4a24-a888-06fc5f4e5fe6/exporter/0.log"
Jul 07 11:53:08.107545 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:08.107517    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8kjb7_cc937ff9-effb-4a24-a888-06fc5f4e5fe6/extractor/0.log"
Jul 07 11:53:10.664113 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:10.664052    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-9k6t8_c4efd1dc-7a63-47d3-a8af-ce5e8e9402e1/manager/0.log"
Jul 07 11:53:10.793112 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:10.793079    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-ht2l6_282ba625-e484-45e0-b00a-962c5431ffba/manager/0.log"
Jul 07 11:53:10.830909 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:10.830878    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-6c8bcdc88b-7wwlc_71ef84c7-9a12-42d5-b95b-410fe33993ef/maas-api/0.log"
Jul 07 11:53:11.004847 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:11.004811    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7bd7d6c999-df6bp_621d0ca4-4868-4d28-b7db-51e4a459338c/manager/0.log"
Jul 07 11:53:11.096213 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:11.096181    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-qrhf8_85974022-4d26-4e54-bda4-3b39cce716cd/manager/0.log"
Jul 07 11:53:11.120759 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:11.120726    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6dc7dd79cd-bxzht_948ae785-e791-40b1-a7c8-029a6e9e3d8e/manager/0.log"
Jul 07 11:53:11.144658 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:11.144629    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6dc7dd79cd-bxzht_948ae785-e791-40b1-a7c8-029a6e9e3d8e/copy-manifests/0.log"
Jul 07 11:53:11.677848 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:11.677820    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59f49fd4f6-lrg8c_397d2fc4-9780-4e14-960c-15519470ad42/payload-pre-processing/0.log"
Jul 07 11:53:11.715560 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:11.715527    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bbcb654f4-96trx_0da2d46b-d50f-44cd-97fa-910881673476/payload-processing/0.log"
Jul 07 11:53:11.728644 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:11.728620    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5dc4cf6785-bflm8_91203aa4-b720-4390-a2c0-5c553ffd79f8/router/0.log"
Jul 07 11:53:12.845298 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:12.845218    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7769b86749-bljkf_b74ff282-2c4a-48e0-a352-fa041935ba7d/manager/0.log"
Jul 07 11:53:13.132602 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:13.132509    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59f49fd4f6-lrg8c_397d2fc4-9780-4e14-960c-15519470ad42/payload-pre-processing/0.log"
Jul 07 11:53:13.179919 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:13.179885    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bbcb654f4-96trx_0da2d46b-d50f-44cd-97fa-910881673476/payload-processing/0.log"
Jul 07 11:53:13.204263 ip-10-0-142-39 kubenswrapper[2570]: I0707 11:53:13.204239    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5dc4cf6785-bflm8_91203aa4-b720-4390-a2c0-5c553ffd79f8/router/0.log"