Jul 07 11:24:53.944725 ip-10-0-135-78 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 07 11:24:53.944737 ip-10-0-135-78 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 07 11:24:53.944747 ip-10-0-135-78 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 07 11:24:53.945074 ip-10-0-135-78 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 07 11:25:05.143093 ip-10-0-135-78 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 07 11:25:05.143108 ip-10-0-135-78 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 407bc31ccdb546e8898dd5204e89086b --
Jul 07 11:26:51.851513 ip-10-0-135-78 systemd[1]: Starting Kubernetes Kubelet...
Jul 07 11:26:53.732081 ip-10-0-135-78 kubenswrapper[2570]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 11:26:53.732081 ip-10-0-135-78 kubenswrapper[2570]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 07 11:26:53.732081 ip-10-0-135-78 kubenswrapper[2570]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 11:26:53.732081 ip-10-0-135-78 kubenswrapper[2570]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 07 11:26:53.732081 ip-10-0-135-78 kubenswrapper[2570]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 11:26:53.733425 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.733333    2570 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 07 11:26:53.738250 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738232    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 11:26:53.738250 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738250    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 11:26:53.738342 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738255    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 11:26:53.738342 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738258    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 11:26:53.738342 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738261    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 11:26:53.738342 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738264    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 11:26:53.738342 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738267    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 11:26:53.738342 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738270    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 11:26:53.738342 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738273    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 11:26:53.738342 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738278    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 11:26:53.738342 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738282    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 11:26:53.738342 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738299    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 11:26:53.738342 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738302    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 11:26:53.738342 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738306    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 11:26:53.738342 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738309    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 11:26:53.738342 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738312    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 11:26:53.738342 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738316    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 11:26:53.738342 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738319    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 11:26:53.738342 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738322    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 11:26:53.738342 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738325    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 11:26:53.738342 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738328    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 11:26:53.738808 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738331    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 11:26:53.738808 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738334    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 11:26:53.738808 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738337    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 11:26:53.738808 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738340    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 11:26:53.738808 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738342    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 11:26:53.738808 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738345    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 11:26:53.738808 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738348    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 11:26:53.738808 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738352    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 11:26:53.738808 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738355    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 11:26:53.738808 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738358    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 11:26:53.738808 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738361    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 11:26:53.738808 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738363    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 11:26:53.738808 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738366    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 11:26:53.738808 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738369    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 11:26:53.738808 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738372    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 11:26:53.738808 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738375    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 11:26:53.738808 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738378    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 11:26:53.738808 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738381    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 11:26:53.738808 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738383    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 11:26:53.738808 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738386    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 11:26:53.739318 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738389    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 11:26:53.739318 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738391    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 11:26:53.739318 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738394    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 11:26:53.739318 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738397    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 11:26:53.739318 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738401    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 11:26:53.739318 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738403    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 11:26:53.739318 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738406    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 11:26:53.739318 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738409    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 11:26:53.739318 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738412    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 11:26:53.739318 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738415    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 11:26:53.739318 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738417    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 11:26:53.739318 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738420    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 11:26:53.739318 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738423    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 11:26:53.739318 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738427    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 11:26:53.739318 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738430    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 11:26:53.739318 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738433    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 11:26:53.739318 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738436    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 11:26:53.739318 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738439    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 11:26:53.739318 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738443    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 11:26:53.739803 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738447    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 11:26:53.739803 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738450    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 11:26:53.739803 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738454    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 11:26:53.739803 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738457    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 11:26:53.739803 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738460    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 11:26:53.739803 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738463    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 11:26:53.739803 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738466    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 11:26:53.739803 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738469    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 11:26:53.739803 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738472    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 11:26:53.739803 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738474    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 11:26:53.739803 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738477    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 11:26:53.739803 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738480    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 11:26:53.739803 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738482    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 11:26:53.739803 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738485    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 11:26:53.739803 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738489    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 11:26:53.739803 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738493    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 11:26:53.739803 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738496    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 11:26:53.739803 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738499    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 11:26:53.739803 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738501    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 11:26:53.739803 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738504    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 11:26:53.740332 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738507    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 11:26:53.740332 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738510    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 11:26:53.740332 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738513    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 11:26:53.740332 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738516    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 11:26:53.740332 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738519    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 11:26:53.740332 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738521    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 11:26:53.740332 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738525    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 11:26:53.740332 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738528    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 11:26:53.740332 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738531    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 11:26:53.740332 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738535    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 11:26:53.740332 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738540    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 11:26:53.740332 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738543    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 11:26:53.740332 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738545    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 11:26:53.740332 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738548    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 11:26:53.740332 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738551    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 11:26:53.740332 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738554    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 11:26:53.740332 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738557    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 11:26:53.740332 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738559    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 11:26:53.740332 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.738562    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 11:26:53.740332 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739344    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 11:26:53.740855 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739351    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 11:26:53.740855 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739355    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 11:26:53.740855 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739358    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 11:26:53.740855 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739361    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 11:26:53.740855 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739364    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 11:26:53.740855 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739367    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 11:26:53.740855 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739370    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 11:26:53.740855 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739373    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 11:26:53.740855 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739376    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 11:26:53.740855 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739381    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 11:26:53.740855 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739385    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 11:26:53.740855 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739389    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 11:26:53.740855 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739393    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 11:26:53.740855 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739396    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 11:26:53.740855 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739399    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 11:26:53.740855 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739403    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 11:26:53.740855 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739406    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 11:26:53.740855 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739408    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 11:26:53.740855 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739412    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 11:26:53.741398 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739415    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 11:26:53.741398 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739418    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 11:26:53.741398 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739421    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 11:26:53.741398 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739424    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 11:26:53.741398 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739427    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 11:26:53.741398 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739429    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 11:26:53.741398 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739432    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 11:26:53.741398 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739435    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 11:26:53.741398 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739438    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 11:26:53.741398 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739440    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 11:26:53.741398 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739444    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 11:26:53.741398 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739447    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 11:26:53.741398 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739449    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 11:26:53.741398 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739452    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 11:26:53.741398 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739455    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 11:26:53.741398 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739458    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 11:26:53.741398 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739461    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 11:26:53.741398 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739463    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 11:26:53.741398 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739467    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 11:26:53.741398 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739470    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 11:26:53.741928 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739473    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 11:26:53.741928 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739476    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 11:26:53.741928 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739481    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 11:26:53.741928 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739484    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 11:26:53.741928 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739487    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 11:26:53.741928 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739490    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 11:26:53.741928 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739493    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 11:26:53.741928 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739495    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 11:26:53.741928 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739498    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 11:26:53.741928 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739503    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 11:26:53.741928 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739506    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 11:26:53.741928 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739510    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 11:26:53.741928 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739513    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 11:26:53.741928 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739515    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 11:26:53.741928 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739518    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 11:26:53.741928 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739521    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 11:26:53.741928 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739524    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 11:26:53.741928 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739527    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 11:26:53.741928 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739529    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 11:26:53.741928 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739532    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 11:26:53.742458 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739535    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 11:26:53.742458 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739537    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 11:26:53.742458 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739540    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 11:26:53.742458 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739543    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 11:26:53.742458 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739546    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 11:26:53.742458 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739550    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 11:26:53.742458 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739553    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 11:26:53.742458 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739556    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 11:26:53.742458 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739559    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 11:26:53.742458 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739561    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 11:26:53.742458 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739565    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 11:26:53.742458 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739568    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 11:26:53.742458 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739570    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 11:26:53.742458 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739573    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 11:26:53.742458 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739576    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 11:26:53.742458 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739580    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 11:26:53.742458 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739582    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 11:26:53.742458 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739585    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 11:26:53.742458 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739588    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 11:26:53.742988 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739591    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 11:26:53.742988 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739594    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 11:26:53.742988 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739597    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 11:26:53.742988 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739600    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 11:26:53.742988 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739603    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 11:26:53.742988 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739607    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 11:26:53.742988 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739610    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 11:26:53.742988 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739613    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 11:26:53.742988 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739616    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 11:26:53.742988 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739619    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 11:26:53.742988 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739621    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 11:26:53.742988 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739624    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 11:26:53.742988 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739627    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 11:26:53.742988 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739630    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 11:26:53.742988 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739633    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 11:26:53.742988 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739635    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 11:26:53.742988 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739638    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 11:26:53.742988 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739641    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 11:26:53.742988 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739643    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 11:26:53.742988 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.739646    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 11:26:53.743522 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740048    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 11:26:53.743522 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740053    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 11:26:53.743522 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740056    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 11:26:53.743522 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740059    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 11:26:53.743522 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740063    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 11:26:53.743522 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740066    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 11:26:53.743522 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740069    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 11:26:53.743522 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740072    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 11:26:53.743522 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740074    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 11:26:53.743522 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740077    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 11:26:53.743522 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740080    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 11:26:53.743522 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740083    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 11:26:53.743522 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740086    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 11:26:53.743522 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740088    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 11:26:53.743522 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740091    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 11:26:53.743522 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740094    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 11:26:53.743522 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740097    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 11:26:53.743522 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740100    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 11:26:53.743522 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740103    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 11:26:53.743522 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740105    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 11:26:53.744037 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740108    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 11:26:53.744037 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740111    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 11:26:53.744037 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740113    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 11:26:53.744037 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740116    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 11:26:53.744037 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740120    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 11:26:53.744037 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740123    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 11:26:53.744037 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740126    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 11:26:53.744037 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740128    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 11:26:53.744037 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740131    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 11:26:53.744037 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740134    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 11:26:53.744037 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740137    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 11:26:53.744037 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740140    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 11:26:53.744037 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740143    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 11:26:53.744037 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740145    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 11:26:53.744037 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740149    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 11:26:53.744037 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740152    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 11:26:53.744037 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740154    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 11:26:53.744037 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740157    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 11:26:53.744037 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740160    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 11:26:53.744037 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740163    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 11:26:53.744565 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740165    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 11:26:53.744565 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740168    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 11:26:53.744565 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740171    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 11:26:53.744565 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740173    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 11:26:53.744565 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740176    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 11:26:53.744565 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740179    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 11:26:53.744565 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740182    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 11:26:53.744565 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740185    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 11:26:53.744565 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740188    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 11:26:53.744565 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740190    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 11:26:53.744565 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740193    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 11:26:53.744565 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740196    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 11:26:53.744565 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740199    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 11:26:53.744565 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740201    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 11:26:53.744565 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740204    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 11:26:53.744565 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740206    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 11:26:53.744565 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740210    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 11:26:53.744565 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740214    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 11:26:53.744565 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740217    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 11:26:53.744565 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740219    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 11:26:53.745082 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740222    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 11:26:53.745082 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740225    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 11:26:53.745082 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740228    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 11:26:53.745082 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740230    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 11:26:53.745082 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740233    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 11:26:53.745082 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740236    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 11:26:53.745082 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740239    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 11:26:53.745082 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740242    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 11:26:53.745082 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740244    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 11:26:53.745082 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740247    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 11:26:53.745082 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740250    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 11:26:53.745082 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740252    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 11:26:53.745082 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740255    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 11:26:53.745082 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740257    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 11:26:53.745082 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740260    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 11:26:53.745082 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740265    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 11:26:53.745082 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740268    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 11:26:53.745082 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740273    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 11:26:53.745082 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740276    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 11:26:53.745603 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740279    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 11:26:53.745603 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740296    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 11:26:53.745603 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740300    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 11:26:53.745603 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740304    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 11:26:53.745603 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740308    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 11:26:53.745603 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740312    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 11:26:53.745603 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740315    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 11:26:53.745603 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740317    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 11:26:53.745603 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740323    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 11:26:53.745603 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740327    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 11:26:53.745603 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740330    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 11:26:53.745603 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740333    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 11:26:53.745603 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740336    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 11:26:53.745603 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740339    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 11:26:53.745603 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740345    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 11:26:53.745603 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740347    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 11:26:53.745603 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740350    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 11:26:53.745603 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740354    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 11:26:53.745603 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740357    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 11:26:53.746097 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.740360    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 11:26:53.746097 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740435    2570 flags.go:64] FLAG: --address="0.0.0.0"
Jul 07 11:26:53.746097 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740442    2570 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 07 11:26:53.746097 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740482    2570 flags.go:64] FLAG: --anonymous-auth="true"
Jul 07 11:26:53.746097 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740488    2570 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 07 11:26:53.746097 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740493    2570 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 07 11:26:53.746097 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740496    2570 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 07 11:26:53.746097 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740501    2570 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 07 11:26:53.746097 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740505    2570 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 07 11:26:53.746097 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740509    2570 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 07 11:26:53.746097 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740512    2570 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 07 11:26:53.746097 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740516    2570 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 07 11:26:53.746097 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740519    2570 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 07 11:26:53.746097 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740523    2570 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 07 11:26:53.746097 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740526    2570 flags.go:64] FLAG: --cgroup-root=""
Jul 07 11:26:53.746097 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740530    2570 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 07 11:26:53.746097 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740534    2570 flags.go:64] FLAG: --client-ca-file=""
Jul 07 11:26:53.746097 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740537    2570 flags.go:64] FLAG: --cloud-provider="external"
Jul 07 11:26:53.746097 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740540    2570 flags.go:64] FLAG: --cluster-dns="[]"
Jul 07 11:26:53.746097 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740545    2570 flags.go:64] FLAG: --cluster-domain=""
Jul 07 11:26:53.746097 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740548    2570 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 07 11:26:53.746097 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740554    2570 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 07 11:26:53.746097 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740557    2570 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 07 11:26:53.746097 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740561    2570 flags.go:64] FLAG: --container-log-max-files="5"
Jul 07 11:26:53.746734 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740565    2570 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 07 11:26:53.746734 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740568    2570 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 07 11:26:53.746734 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740572    2570 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 07 11:26:53.746734 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740577    2570 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 07 11:26:53.746734 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740580    2570 flags.go:64] FLAG: --contention-profiling="false"
Jul 07 11:26:53.746734 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740584    2570 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 07 11:26:53.746734 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740587    2570 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 07 11:26:53.746734 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740590    2570 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 07 11:26:53.746734 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740594    2570 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 07 11:26:53.746734 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740598    2570 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 07 11:26:53.746734 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740602    2570 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 07 11:26:53.746734 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740605    2570 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 07 11:26:53.746734 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740608    2570 flags.go:64] FLAG: --enable-load-reader="false"
Jul 07 11:26:53.746734 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740611    2570 flags.go:64] FLAG: --enable-server="true"
Jul 07 11:26:53.746734 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740615    2570 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 07 11:26:53.746734 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740620    2570 flags.go:64] FLAG: --event-burst="100"
Jul 07 11:26:53.746734 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740623    2570 flags.go:64] FLAG: --event-qps="50"
Jul 07 11:26:53.746734 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740626    2570 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 07 11:26:53.746734 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740630    2570 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 07 11:26:53.746734 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740633    2570 flags.go:64] FLAG: --eviction-hard=""
Jul 07 11:26:53.746734 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740637    2570 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 07 11:26:53.746734 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740640    2570 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 07 11:26:53.746734 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740643    2570 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 07 11:26:53.746734 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740646    2570 flags.go:64] FLAG: --eviction-soft=""
Jul 07 11:26:53.746734 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740649    2570 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 07 11:26:53.747377 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740652    2570 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 07 11:26:53.747377 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740656    2570 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 07 11:26:53.747377 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740659    2570 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 07 11:26:53.747377 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740662    2570 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 07 11:26:53.747377 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740667    2570 flags.go:64] FLAG: --fail-swap-on="true"
Jul 07 11:26:53.747377 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740670    2570 flags.go:64] FLAG: --feature-gates=""
Jul 07 11:26:53.747377 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740674    2570 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 07 11:26:53.747377 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740677    2570 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 07 11:26:53.747377 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740681    2570 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 07 11:26:53.747377 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740684    2570 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 07 11:26:53.747377 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740689    2570 flags.go:64] FLAG: --healthz-port="10248"
Jul 07 11:26:53.747377 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740692    2570 flags.go:64] FLAG: --help="false"
Jul 07 11:26:53.747377 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740695    2570 flags.go:64] FLAG: --hostname-override="ip-10-0-135-78.ec2.internal"
Jul 07 11:26:53.747377 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740699    2570 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 07 11:26:53.747377 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740702    2570 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 07 11:26:53.747377 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740705    2570 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 07 11:26:53.747377 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740709    2570 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 07 11:26:53.747377 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740713    2570 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 07 11:26:53.747377 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740716    2570 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 07 11:26:53.747377 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740719    2570 flags.go:64] FLAG: --image-service-endpoint=""
Jul 07 11:26:53.747377 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740722    2570 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 07 11:26:53.747377 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740725    2570 flags.go:64] FLAG: --kube-api-burst="100"
Jul 07 11:26:53.747377 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740728    2570 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 07 11:26:53.748141 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740731    2570 flags.go:64] FLAG: --kube-api-qps="50"
Jul 07 11:26:53.748141 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740734    2570 flags.go:64] FLAG: --kube-reserved=""
Jul 07 11:26:53.748141 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740737    2570 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 07 11:26:53.748141 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740740    2570 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 07 11:26:53.748141 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740744    2570 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 07 11:26:53.748141 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740747    2570 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 07 11:26:53.748141 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740750    2570 flags.go:64] FLAG: --lock-file=""
Jul 07 11:26:53.748141 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740753    2570 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 07 11:26:53.748141 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740756    2570 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 07 11:26:53.748141 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740759    2570 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 07 11:26:53.748141 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740769    2570 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 07 11:26:53.748141 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740772    2570 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 07 11:26:53.748141 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740775    2570 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 07 11:26:53.748141 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740779    2570 flags.go:64] FLAG: --logging-format="text"
Jul 07 11:26:53.748141 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740782    2570 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 07 11:26:53.748141 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740786    2570 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 07 11:26:53.748141 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740789    2570 flags.go:64] FLAG: --manifest-url=""
Jul 07 11:26:53.748141 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740792    2570 flags.go:64] FLAG: --manifest-url-header=""
Jul 07 11:26:53.748141 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740796    2570 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 07 11:26:53.748141 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740804    2570 flags.go:64] FLAG: --max-open-files="1000000"
Jul 07 11:26:53.748141 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740809    2570 flags.go:64] FLAG: --max-pods="110"
Jul 07 11:26:53.748141 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740812    2570 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 07 11:26:53.748141 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740815    2570 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 07 11:26:53.748141 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740818    2570 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 07 11:26:53.748141 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740821    2570 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 07 11:26:53.749107 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740825    2570 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 07 11:26:53.749107 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740829    2570 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 07 11:26:53.749107 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740832    2570 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 07 11:26:53.749107 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740840    2570 flags.go:64] FLAG: --node-status-max-images="50"
Jul 07 11:26:53.749107 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740843    2570 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 07 11:26:53.749107 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740846    2570 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 07 11:26:53.749107 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740849    2570 flags.go:64] FLAG: --pod-cidr=""
Jul 07 11:26:53.749107 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740853    2570 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 07 11:26:53.749107 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740859    2570 flags.go:64] FLAG: --pod-manifest-path=""
Jul 07 11:26:53.749107 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740863    2570 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 07 11:26:53.749107 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740866    2570 flags.go:64] FLAG: --pods-per-core="0"
Jul 07 11:26:53.749107 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740869    2570 flags.go:64] FLAG: --port="10250"
Jul 07 11:26:53.749107 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740873    2570 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 07 11:26:53.749107 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740876    2570 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e0f60d6e8af0c7f0"
Jul 07 11:26:53.749107 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740879    2570 flags.go:64] FLAG: --qos-reserved=""
Jul 07 11:26:53.749107 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740882    2570 flags.go:64] FLAG: --read-only-port="10255"
Jul 07 11:26:53.749107 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740885    2570 flags.go:64] FLAG: --register-node="true"
Jul 07 11:26:53.749107 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740889    2570 flags.go:64] FLAG: --register-with-taints=""
Jul 07 11:26:53.749107 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740893    2570 flags.go:64] FLAG: --registry-burst="10"
Jul 07 11:26:53.749107 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740896    2570 flags.go:64] FLAG: --registry-qps="5"
Jul 07 11:26:53.749107 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740901    2570 flags.go:64] FLAG: --reserved-cpus=""
Jul 07 11:26:53.749107 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740904    2570 flags.go:64] FLAG: --reserved-memory=""
Jul 07 11:26:53.749107 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740908    2570 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 07 11:26:53.749107 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740912    2570 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 07 11:26:53.749107 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740915    2570 flags.go:64] FLAG: --rotate-certificates="false"
Jul 07 11:26:53.749809 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740918    2570 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 07 11:26:53.749809 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740921    2570 flags.go:64] FLAG: --runonce="false"
Jul 07 11:26:53.749809 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740925    2570 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 07 11:26:53.749809 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740928    2570 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 07 11:26:53.749809 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740931    2570 flags.go:64] FLAG: --seccomp-default="false"
Jul 07 11:26:53.749809 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740934    2570 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 07 11:26:53.749809 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740938    2570 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 07 11:26:53.749809 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740941    2570 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 07 11:26:53.749809 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740956    2570 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 07 11:26:53.749809 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740961    2570 flags.go:64] FLAG: --storage-driver-password="root"
Jul 07 11:26:53.749809 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740965    2570 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 07 11:26:53.749809 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740968    2570 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 07 11:26:53.749809 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740971    2570 flags.go:64] FLAG: --storage-driver-user="root"
Jul 07 11:26:53.749809 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740974    2570 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 07 11:26:53.749809 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740977    2570 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 07 11:26:53.749809 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740981    2570 flags.go:64] FLAG: --system-cgroups=""
Jul 07 11:26:53.749809 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740984    2570 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 07 11:26:53.749809 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740989    2570 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 07 11:26:53.749809 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740992    2570 flags.go:64] FLAG: --tls-cert-file=""
Jul 07 11:26:53.749809 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.740995    2570 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 07 11:26:53.749809 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.741000    2570 flags.go:64] FLAG: --tls-min-version=""
Jul 07 11:26:53.749809 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.741002    2570 flags.go:64] FLAG: --tls-private-key-file=""
Jul 07 11:26:53.749809 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.741005    2570 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 07 11:26:53.749809 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.741008    2570 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 07 11:26:53.749809 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.741011    2570 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 07 11:26:53.750495 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.741015    2570 flags.go:64] FLAG: --v="2"
Jul 07 11:26:53.750495 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.741020    2570 flags.go:64] FLAG: --version="false"
Jul 07 11:26:53.750495 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.741026    2570 flags.go:64] FLAG: --vmodule=""
Jul 07 11:26:53.750495 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.741030    2570 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 07 11:26:53.750495 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.741034    2570 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 07 11:26:53.750495 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741159    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 11:26:53.750495 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741163    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 11:26:53.750495 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741166    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 11:26:53.750495 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741170    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 11:26:53.750495 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741173    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 11:26:53.750495 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741176    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 11:26:53.750495 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741179    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 11:26:53.750495 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741182    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 11:26:53.750495 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741185    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 11:26:53.750495 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741187    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 11:26:53.750495 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741190    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 11:26:53.750495 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741193    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 11:26:53.750495 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741195    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 11:26:53.750495 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741198    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 11:26:53.750495 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741201    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 11:26:53.750495 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741204    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 11:26:53.751053 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741206    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 11:26:53.751053 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741209    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 11:26:53.751053 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741212    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 11:26:53.751053 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741215    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 11:26:53.751053 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741218    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 11:26:53.751053 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741221    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 11:26:53.751053 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741223    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 11:26:53.751053 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741226    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 11:26:53.751053 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741229    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 11:26:53.751053 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741232    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 11:26:53.751053 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741234    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 11:26:53.751053 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741237    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 11:26:53.751053 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741240    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 11:26:53.751053 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741245    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 11:26:53.751053 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741248    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 11:26:53.751053 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741251    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 11:26:53.751053 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741255    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 11:26:53.751053 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741259    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 11:26:53.751053 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741262    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 11:26:53.751053 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741265    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 11:26:53.751606 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741268    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 11:26:53.751606 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741270    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 11:26:53.751606 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741273    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 11:26:53.751606 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741276    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 11:26:53.751606 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741279    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 11:26:53.751606 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741281    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 11:26:53.751606 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741305    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 11:26:53.751606 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741309    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 11:26:53.751606 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741312    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 11:26:53.751606 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741314    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 11:26:53.751606 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741317    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 11:26:53.751606 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741320    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 11:26:53.751606 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741322    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 11:26:53.751606 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741325    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 11:26:53.751606 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741328    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 11:26:53.751606 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741331    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 11:26:53.751606 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741334    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 11:26:53.751606 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741336    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 11:26:53.751606 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741339    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 11:26:53.751606 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741342    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 11:26:53.752135 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741345    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 11:26:53.752135 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741347    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 11:26:53.752135 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741350    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 11:26:53.752135 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741353    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 11:26:53.752135 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741356    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 11:26:53.752135 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741360    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 11:26:53.752135 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741364    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 11:26:53.752135 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741367    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 11:26:53.752135 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741370    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 11:26:53.752135 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741372    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 11:26:53.752135 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741375    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 11:26:53.752135 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741378    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 11:26:53.752135 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741381    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 11:26:53.752135 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741385    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 11:26:53.752135 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741389    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 11:26:53.752135 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741393    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 11:26:53.752135 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741396    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 11:26:53.752135 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741399    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 11:26:53.752135 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741402    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 11:26:53.752662 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741405    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 11:26:53.752662 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741408    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 11:26:53.752662 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741412    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 11:26:53.752662 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741414    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 11:26:53.752662 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741417    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 11:26:53.752662 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741420    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 11:26:53.752662 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741423    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 11:26:53.752662 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741425    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 11:26:53.752662 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741428    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 11:26:53.752662 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741431    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 11:26:53.752662 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741434    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 11:26:53.752662 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741437    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 11:26:53.752662 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741439    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 11:26:53.752662 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741442    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 11:26:53.752662 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741445    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 11:26:53.752662 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741448    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 11:26:53.752662 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741451    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 11:26:53.752662 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741453    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 11:26:53.752662 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741457    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 11:26:53.752662 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741460    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 11:26:53.753220 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741469    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 11:26:53.753220 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741472    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 11:26:53.753220 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741475    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 11:26:53.753220 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.741477    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 11:26:53.753220 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.741483    2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 11:26:53.753220 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.750424    2570 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 07 11:26:53.753220 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.750443    2570 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 07 11:26:53.753220 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.750463    2570 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 07 11:26:53.753220 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.750469    2570 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 07 11:26:53.753220 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750532    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 11:26:53.753220 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750538    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 11:26:53.753220 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750543    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 11:26:53.753220 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750546    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 11:26:53.753220 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750549    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 11:26:53.753220 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750552    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 11:26:53.753647 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750555    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 11:26:53.753647 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750558    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 11:26:53.753647 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750561    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 11:26:53.753647 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750564    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 11:26:53.753647 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750567    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 11:26:53.753647 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750570    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 11:26:53.753647 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750573    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 11:26:53.753647 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750576    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 11:26:53.753647 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750579    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 11:26:53.753647 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750582    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 11:26:53.753647 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750584    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 11:26:53.753647 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750587    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 11:26:53.753647 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750591    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 11:26:53.753647 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750594    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 11:26:53.753647 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750597    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 11:26:53.753647 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750601    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 11:26:53.753647 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750605    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 11:26:53.753647 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750608    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 11:26:53.753647 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750611    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 11:26:53.753647 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750614    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 11:26:53.754180 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750617    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 11:26:53.754180 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750620    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 11:26:53.754180 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750623    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 11:26:53.754180 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750627    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 11:26:53.754180 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750630    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 11:26:53.754180 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750633    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 11:26:53.754180 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750636    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 11:26:53.754180 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750639    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 11:26:53.754180 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750642    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 11:26:53.754180 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750645    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 11:26:53.754180 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750648    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 11:26:53.754180 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750650    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 11:26:53.754180 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750653    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 11:26:53.754180 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750656    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 11:26:53.754180 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750659    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 11:26:53.754180 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750662    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 11:26:53.754180 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750664    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 11:26:53.754180 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750667    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 11:26:53.754180 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750670    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 11:26:53.754180 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750673    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 11:26:53.754726 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750676    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 11:26:53.754726 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750679    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 11:26:53.754726 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750682    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 11:26:53.754726 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750685    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 11:26:53.754726 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750688    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 11:26:53.754726 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750691    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 11:26:53.754726 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750694    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 11:26:53.754726 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750697    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 11:26:53.754726 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750701    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 11:26:53.754726 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750705    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 11:26:53.754726 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750709    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 11:26:53.754726 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750713    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 11:26:53.754726 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750716    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 11:26:53.754726 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750719    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 11:26:53.754726 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750722    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 11:26:53.754726 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750725    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 11:26:53.754726 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750729    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 11:26:53.754726 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750731    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 11:26:53.754726 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750734    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 11:26:53.755219 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750737    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 11:26:53.755219 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750740    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 11:26:53.755219 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750743    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 11:26:53.755219 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750746    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 11:26:53.755219 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750749    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 11:26:53.755219 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750751    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 11:26:53.755219 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750754    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 11:26:53.755219 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750757    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 11:26:53.755219 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750759    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 11:26:53.755219 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750762    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 11:26:53.755219 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750766    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 11:26:53.755219 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750770    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 11:26:53.755219 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750773    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 11:26:53.755219 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750776    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 11:26:53.755219 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750780    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 11:26:53.755219 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750784    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 11:26:53.755219 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750787    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 11:26:53.755219 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750790    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 11:26:53.755219 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750792    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 11:26:53.755753 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750795    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 11:26:53.755753 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750798    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 11:26:53.755753 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750801    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 11:26:53.755753 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750803    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 11:26:53.755753 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750806    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 11:26:53.755753 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750809    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 11:26:53.755753 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750811    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 11:26:53.755753 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750814    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 11:26:53.755753 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750817    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 11:26:53.755753 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750820    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 11:26:53.755753 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750823    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 11:26:53.755753 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750826    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 11:26:53.755753 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750829    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 11:26:53.755753 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750832    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 11:26:53.755753 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750835    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 11:26:53.756140 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.750840    2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 11:26:53.756140 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750965    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 11:26:53.756140 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750970    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 11:26:53.756140 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750974    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 11:26:53.756140 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750977    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 11:26:53.756140 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750980    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 11:26:53.756140 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750984    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 11:26:53.756140 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750987    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 11:26:53.756140 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750990    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 11:26:53.756140 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750993    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 11:26:53.756140 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.750996    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 11:26:53.756140 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751000    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 11:26:53.756140 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751003    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 11:26:53.756140 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751006    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 11:26:53.756140 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751008    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 11:26:53.756565 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751011    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 11:26:53.756565 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751014    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 11:26:53.756565 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751017    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 11:26:53.756565 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751020    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 11:26:53.756565 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751023    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 11:26:53.756565 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751026    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 11:26:53.756565 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751029    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 11:26:53.756565 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751031    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 11:26:53.756565 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751034    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 11:26:53.756565 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751037    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 11:26:53.756565 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751040    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 11:26:53.756565 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751043    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 11:26:53.756565 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751046    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 11:26:53.756565 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751049    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 11:26:53.756565 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751052    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 11:26:53.756565 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751055    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 11:26:53.756565 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751058    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 11:26:53.756565 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751060    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 11:26:53.756565 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751063    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 11:26:53.756565 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751066    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 11:26:53.757102 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751069    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 11:26:53.757102 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751072    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 11:26:53.757102 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751075    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 11:26:53.757102 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751078    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 11:26:53.757102 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751080    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 11:26:53.757102 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751083    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 11:26:53.757102 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751086    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 11:26:53.757102 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751088    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 11:26:53.757102 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751091    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 11:26:53.757102 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751094    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 11:26:53.757102 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751097    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 11:26:53.757102 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751101    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 11:26:53.757102 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751104    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 11:26:53.757102 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751107    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 11:26:53.757102 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751109    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 11:26:53.757102 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751112    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 11:26:53.757102 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751115    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 11:26:53.757102 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751117    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 11:26:53.757102 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751120    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 11:26:53.757615 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751123    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 11:26:53.757615 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751126    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 11:26:53.757615 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751129    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 11:26:53.757615 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751131    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 11:26:53.757615 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751134    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 11:26:53.757615 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751137    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 11:26:53.757615 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751140    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 11:26:53.757615 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751142    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 11:26:53.757615 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751145    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 11:26:53.757615 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751148    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 11:26:53.757615 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751152    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 11:26:53.757615 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751156    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 11:26:53.757615 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751159    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 11:26:53.757615 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751162    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 11:26:53.757615 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751166    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 11:26:53.757615 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751170    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 11:26:53.757615 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751173    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 11:26:53.757615 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751176    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 11:26:53.757615 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751179    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 11:26:53.758110 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751182    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 11:26:53.758110 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751184    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 11:26:53.758110 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751187    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 11:26:53.758110 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751190    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 11:26:53.758110 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751193    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 11:26:53.758110 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751196    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 11:26:53.758110 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751200    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 11:26:53.758110 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751202    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 11:26:53.758110 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751205    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 11:26:53.758110 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751208    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 11:26:53.758110 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751210    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 11:26:53.758110 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751213    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 11:26:53.758110 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751216    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 11:26:53.758110 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751219    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 11:26:53.758110 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751223    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 11:26:53.758110 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751226    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 11:26:53.758110 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751229    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 11:26:53.758110 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751232    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 11:26:53.758110 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751235    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 11:26:53.758110 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751238    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 11:26:53.758657 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751240    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 11:26:53.758657 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751243    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 11:26:53.758657 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751246    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 11:26:53.758657 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751249    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 11:26:53.758657 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751251    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 11:26:53.758657 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751254    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 11:26:53.758657 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:53.751257    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 11:26:53.758657 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.751262    2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 11:26:53.758657 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.751435    2570 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 07 11:26:53.758657 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.754509    2570 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 07 11:26:53.758657 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.756483    2570 server.go:1019] "Starting client certificate rotation"
Jul 07 11:26:53.758657 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.756586    2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 07 11:26:53.758657 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.756620    2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 07 11:26:53.759359 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.759326    2570 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 07 11:26:53.760544 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.760525    2570 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 07 11:26:53.769810 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.769790    2570 log.go:25] "Validated CRI v1 runtime API"
Jul 07 11:26:53.774540 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.774521    2570 log.go:25] "Validated CRI v1 image API"
Jul 07 11:26:53.777924 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.777898    2570 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 07 11:26:53.780923 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.780901    2570 fs.go:135] Filesystem UUIDs: map[1a23b6c8-979f-4515-8f39-5116c24b9188:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 ca6eb703-7f6b-41e5-b56f-323b295d88fe:/dev/nvme0n1p4]
Jul 07 11:26:53.781011 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.780921    2570 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 07 11:26:53.787842 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.787709    2570 manager.go:216] Machine: {Timestamp:2026-07-07 11:26:53.787122657 +0000 UTC m=+1.760763062 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3007698 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec28db41d1e6f7b9784782d9e140a0e0 SystemUUID:ec28db41-d1e6-f7b9-7847-82d9e140a0e0 BootID:407bc31c-cdb5-46e8-898d-d5204e89086b Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:9a:b7:56:75:8f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:9a:b7:56:75:8f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:32:19:b2:0c:cb:af Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 07 11:26:53.787842 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.787827    2570 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 07 11:26:53.788022 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.787949    2570 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 07 11:26:53.790416 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.790394    2570 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 07 11:26:53.790903 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.790873    2570 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 07 11:26:53.791042 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.790905    2570 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-78.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 07 11:26:53.791619 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.791608    2570 topology_manager.go:138] "Creating topology manager with none policy"
Jul 07 11:26:53.791654 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.791622    2570 container_manager_linux.go:309] "Creating device plugin manager"
Jul 07 11:26:53.791654 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.791630    2570 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 07 11:26:53.792513 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.792502    2570 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 07 11:26:53.793835 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.793823    2570 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 07 11:26:53.798038 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.798025    2570 state_mem.go:36] "Initialized new in-memory state store"
Jul 07 11:26:53.798165 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.798155    2570 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 07 11:26:53.798206 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.798189    2570 kubelet.go:486] "Attempting to sync node with API server"
Jul 07 11:26:53.798206 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.798199    2570 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 07 11:26:53.798261 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.798211    2570 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 07 11:26:53.798261 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.798221    2570 kubelet.go:398] "Adding apiserver pod source"
Jul 07 11:26:53.798261 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.798231    2570 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 07 11:26:53.798650 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.798632    2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 07 11:26:53.798650 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.798652    2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 07 11:26:53.801131 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.801112    2570 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 07 11:26:53.803244 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.803230    2570 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 07 11:26:53.803320 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.803263    2570 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 07 11:26:53.804528 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.804512    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 07 11:26:53.804620 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.804532    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 07 11:26:53.804620 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.804542    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 07 11:26:53.804620 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.804552    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 07 11:26:53.804620 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.804565    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 07 11:26:53.804620 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.804574    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 07 11:26:53.804620 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.804589    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 07 11:26:53.804620 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.804599    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 07 11:26:53.804620 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.804610    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 07 11:26:53.804620 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.804619    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 07 11:26:53.804894 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.804634    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 07 11:26:53.804894 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.804643    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 07 11:26:53.805559 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.805542    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 07 11:26:53.805559 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.805560    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 07 11:26:53.809657 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.809642    2570 server.go:1268] "Started kubelet"
Jul 07 11:26:53.809749 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.809702    2570 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 07 11:26:53.809853 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.809802    2570 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 07 11:26:53.809909 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.809879    2570 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 07 11:26:53.810305 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:53.810285    2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-78.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 07 11:26:53.810377 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:53.810356    2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 07 11:26:53.810417 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.810384    2570 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-78.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 07 11:26:53.811692 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.811677    2570 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 07 11:26:53.812491 ip-10-0-135-78 systemd[1]: Started Kubernetes Kubelet.
Jul 07 11:26:53.813702 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.813677    2570 server.go:310] "Adding debug handlers to kubelet server"
Jul 07 11:26:53.816895 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:53.816873    2570 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 07 11:26:53.818562 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:53.817501    2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-78.ec2.internal.18bffe53e92a5069  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-78.ec2.internal,UID:ip-10-0-135-78.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-78.ec2.internal,},FirstTimestamp:2026-07-07 11:26:53.809610857 +0000 UTC m=+1.783251261,LastTimestamp:2026-07-07 11:26:53.809610857 +0000 UTC m=+1.783251261,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-78.ec2.internal,}"
Jul 07 11:26:53.820580 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.820561    2570 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 07 11:26:53.820675 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.820636    2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 07 11:26:53.822807 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:53.822044    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-78.ec2.internal\" not found"
Jul 07 11:26:53.822807 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.821807    2570 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 07 11:26:53.822807 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.822433    2570 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 07 11:26:53.822807 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.822707    2570 reconstruct.go:97] "Volume reconstruction finished"
Jul 07 11:26:53.822807 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.822721    2570 reconciler.go:29] "Reconciler: start to sync state"
Jul 07 11:26:53.822807 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.822780    2570 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 07 11:26:53.824006 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.823966    2570 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 07 11:26:53.824006 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.824004    2570 factory.go:55] Registering systemd factory
Jul 07 11:26:53.824140 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.824019    2570 factory.go:223] Registration of the systemd container factory successfully
Jul 07 11:26:53.824393 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.824375    2570 factory.go:153] Registering CRI-O factory
Jul 07 11:26:53.824393 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.824391    2570 factory.go:223] Registration of the crio container factory successfully
Jul 07 11:26:53.824604 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.824418    2570 factory.go:103] Registering Raw factory
Jul 07 11:26:53.824604 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.824436    2570 manager.go:1196] Started watching for new ooms in manager
Jul 07 11:26:53.824954 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.824938    2570 manager.go:349] Starting recovery of all containers
Jul 07 11:26:53.831927 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:53.830105    2570 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 07 11:26:53.831927 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:53.830441    2570 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-78.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 07 11:26:53.836815 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.836797    2570 manager.go:354] Recovery completed
Jul 07 11:26:53.840602 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.840581    2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-k8k9t"
Jul 07 11:26:53.841846 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.841829    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 11:26:53.843961 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.843944    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-78.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 11:26:53.844024 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.843974    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-78.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 11:26:53.844024 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.843985    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-78.ec2.internal" event="NodeHasSufficientPID"
Jul 07 11:26:53.845103 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.845085    2570 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 07 11:26:53.845103 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.845102    2570 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 07 11:26:53.845231 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.845119    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 11:26:53.845231 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.845124    2570 state_mem.go:36] "Initialized new in-memory state store"
Jul 07 11:26:53.846250 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:53.846144    2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-78.ec2.internal.18bffe53eb3672cd  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-78.ec2.internal,UID:ip-10-0-135-78.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-78.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-78.ec2.internal,},FirstTimestamp:2026-07-07 11:26:53.843960525 +0000 UTC m=+1.817600933,LastTimestamp:2026-07-07 11:26:53.843960525 +0000 UTC m=+1.817600933,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-78.ec2.internal,}"
Jul 07 11:26:53.846484 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.846450    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-78.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 11:26:53.846540 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.846509    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-78.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 11:26:53.846540 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.846525    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-78.ec2.internal" event="NodeHasSufficientPID"
Jul 07 11:26:53.847858 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.847844    2570 policy_none.go:49] "None policy: Start"
Jul 07 11:26:53.847912 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.847862    2570 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 07 11:26:53.847912 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.847873    2570 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 07 11:26:53.849623 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.849607    2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-k8k9t"
Jul 07 11:26:53.850056 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.850042    2570 policy_none.go:47] "Start"
Jul 07 11:26:53.891791 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.891776    2570 manager.go:337] "Starting Device Plugin manager"
Jul 07 11:26:53.891863 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:53.891811    2570 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 07 11:26:53.891863 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.891822    2570 server.go:87] "Starting device plugin registration server"
Jul 07 11:26:53.892156 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.892140    2570 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 07 11:26:53.892228 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.892156    2570 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 07 11:26:53.892313 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.892243    2570 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 07 11:26:53.892384 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.892348    2570 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 07 11:26:53.892384 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.892358    2570 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 07 11:26:53.893009 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:53.892988    2570 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 07 11:26:53.893090 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:53.893037    2570 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-78.ec2.internal\" not found"
Jul 07 11:26:53.930197 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.930157    2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 07 11:26:53.938550 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.931505    2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 07 11:26:53.938550 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.931553    2570 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 07 11:26:53.938550 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.931578    2570 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 07 11:26:53.938550 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:53.931613    2570 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 07 11:26:53.938550 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.934556    2570 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 11:26:53.993195 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.993103    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 11:26:53.994856 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.994834    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-78.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 11:26:53.994966 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.994878    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-78.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 11:26:53.994966 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.994895    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-78.ec2.internal" event="NodeHasSufficientPID"
Jul 07 11:26:53.994966 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:53.994930    2570 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-78.ec2.internal"
Jul 07 11:26:54.005562 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:54.005537    2570 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-78.ec2.internal"
Jul 07 11:26:54.005666 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:54.005568    2570 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-78.ec2.internal\": node \"ip-10-0-135-78.ec2.internal\" not found"
Jul 07 11:26:54.031871 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:54.031832    2570 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-78.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-78.ec2.internal"]
Jul 07 11:26:54.032050 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:54.031907    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 11:26:54.032906 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:54.032885    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-78.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 11:26:54.033012 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:54.032922    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-78.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 11:26:54.033012 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:54.032939    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-78.ec2.internal" event="NodeHasSufficientPID"
Jul 07 11:26:54.035396 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:54.035380    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 11:26:54.035566 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:54.035550    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-78.ec2.internal"
Jul 07 11:26:54.035629 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:54.035588    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 11:26:54.036176 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:54.036160    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-78.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 11:26:54.036247 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:54.036193    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-78.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 11:26:54.036247 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:54.036205    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-78.ec2.internal" event="NodeHasSufficientPID"
Jul 07 11:26:54.036368 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:54.036161    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-78.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 11:26:54.036368 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:54.036277    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-78.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 11:26:54.036368 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:54.036298    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-78.ec2.internal" event="NodeHasSufficientPID"
Jul 07 11:26:54.038815 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:54.038792    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-78.ec2.internal"
Jul 07 11:26:54.038874 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:54.038823    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 11:26:54.039587 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:54.039572    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-78.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 11:26:54.039662 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:54.039603    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-78.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 11:26:54.039662 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:54.039618    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-78.ec2.internal" event="NodeHasSufficientPID"
Jul 07 11:26:54.044688 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:54.044668    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-78.ec2.internal\" not found"
Jul 07 11:26:54.064315 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:54.064287    2570 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-78.ec2.internal\" not found" node="ip-10-0-135-78.ec2.internal"
Jul 07 11:26:54.069347 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:54.069327    2570 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-78.ec2.internal\" not found" node="ip-10-0-135-78.ec2.internal"
Jul 07 11:26:54.124953 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:54.124920    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c9aa93f97f9dd2447825488095d15a9f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-78.ec2.internal\" (UID: \"c9aa93f97f9dd2447825488095d15a9f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-78.ec2.internal"
Jul 07 11:26:54.124953 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:54.124961    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c9aa93f97f9dd2447825488095d15a9f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-78.ec2.internal\" (UID: \"c9aa93f97f9dd2447825488095d15a9f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-78.ec2.internal"
Jul 07 11:26:54.125191 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:54.124992    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4f2009fc30615ef1da5c98996e493d3f-config\") pod \"kube-apiserver-proxy-ip-10-0-135-78.ec2.internal\" (UID: \"4f2009fc30615ef1da5c98996e493d3f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-78.ec2.internal"
Jul 07 11:26:54.145340 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:54.145312    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-78.ec2.internal\" not found"
Jul 07 11:26:54.225597 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:54.225559    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c9aa93f97f9dd2447825488095d15a9f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-78.ec2.internal\" (UID: \"c9aa93f97f9dd2447825488095d15a9f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-78.ec2.internal"
Jul 07 11:26:54.225683 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:54.225656    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c9aa93f97f9dd2447825488095d15a9f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-78.ec2.internal\" (UID: \"c9aa93f97f9dd2447825488095d15a9f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-78.ec2.internal"
Jul 07 11:26:54.225738 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:54.225700    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c9aa93f97f9dd2447825488095d15a9f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-78.ec2.internal\" (UID: \"c9aa93f97f9dd2447825488095d15a9f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-78.ec2.internal"
Jul 07 11:26:54.225775 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:54.225753    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4f2009fc30615ef1da5c98996e493d3f-config\") pod \"kube-apiserver-proxy-ip-10-0-135-78.ec2.internal\" (UID: \"4f2009fc30615ef1da5c98996e493d3f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-78.ec2.internal"
Jul 07 11:26:54.225814 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:54.225793    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4f2009fc30615ef1da5c98996e493d3f-config\") pod \"kube-apiserver-proxy-ip-10-0-135-78.ec2.internal\" (UID: \"4f2009fc30615ef1da5c98996e493d3f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-78.ec2.internal"
Jul 07 11:26:54.225814 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:54.225806    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c9aa93f97f9dd2447825488095d15a9f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-78.ec2.internal\" (UID: \"c9aa93f97f9dd2447825488095d15a9f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-78.ec2.internal"
Jul 07 11:26:54.246128 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:54.246032    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-78.ec2.internal\" not found"
Jul 07 11:26:54.347461 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:54.347422    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-78.ec2.internal\" not found"
Jul 07 11:26:54.368151 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:54.368115    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-78.ec2.internal"
Jul 07 11:26:54.373817 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:54.373553    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-78.ec2.internal"
Jul 07 11:26:54.448981 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:54.448926    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-78.ec2.internal\" not found"
Jul 07 11:26:54.550404 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:54.550324    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-78.ec2.internal\" not found"
Jul 07 11:26:54.651306 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:54.651259    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-78.ec2.internal\" not found"
Jul 07 11:26:54.752104 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:54.752053    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-78.ec2.internal\" not found"
Jul 07 11:26:54.756536 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:54.756515    2570 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 07 11:26:54.756704 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:54.756686    2570 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 07 11:26:54.853382 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:54.853273    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-78.ec2.internal\" not found"
Jul 07 11:26:54.853382 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:54.853294    2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 11:21:54 +0000 UTC" deadline="2028-03-14 14:46:14.508855832 +0000 UTC"
Jul 07 11:26:54.853382 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:54.853366    2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14787h19m19.655496131s"
Jul 07 11:26:54.954421 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:54.954353    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-78.ec2.internal\" not found"
Jul 07 11:26:54.972668 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:54.972640    2570 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 11:26:55.055533 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:55.055498    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-78.ec2.internal\" not found"
Jul 07 11:26:55.116118 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.116091    2570 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 11:26:55.123699 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.123674    2570 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-78.ec2.internal"
Jul 07 11:26:55.128276 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:55.128228    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9aa93f97f9dd2447825488095d15a9f.slice/crio-db88debb254d9d89eaed6846daf44b32af3680a893dab8f7d6f9c35c5f571cf2 WatchSource:0}: Error finding container db88debb254d9d89eaed6846daf44b32af3680a893dab8f7d6f9c35c5f571cf2: Status 404 returned error can't find the container with id db88debb254d9d89eaed6846daf44b32af3680a893dab8f7d6f9c35c5f571cf2
Jul 07 11:26:55.133387 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.133371    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 11:26:55.134591 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:55.134569    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f2009fc30615ef1da5c98996e493d3f.slice/crio-9a943cb5da943f857849307cb48ff32760b49ce839664d261d1bc419cb3584fa WatchSource:0}: Error finding container 9a943cb5da943f857849307cb48ff32760b49ce839664d261d1bc419cb3584fa: Status 404 returned error can't find the container with id 9a943cb5da943f857849307cb48ff32760b49ce839664d261d1bc419cb3584fa
Jul 07 11:26:55.138402 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.138382    2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 07 11:26:55.140898 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.140880    2570 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-78.ec2.internal"
Jul 07 11:26:55.150154 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.150135    2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 07 11:26:55.399109 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.399024    2570 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 11:26:55.802144 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.802040    2570 apiserver.go:52] "Watching apiserver"
Jul 07 11:26:55.812556 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.812527    2570 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 07 11:26:55.813741 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.813711    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-lj4rs","openshift-multus/network-metrics-daemon-ppmbz","openshift-network-operator/iptables-alerter-vdrl5","openshift-ovn-kubernetes/ovnkube-node-pm4tw","kube-system/global-pull-secret-syncer-8f5jf","kube-system/konnectivity-agent-7tk6n","kube-system/kube-apiserver-proxy-ip-10-0-135-78.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lmxhq","openshift-image-registry/node-ca-qh2dm","openshift-multus/multus-additional-cni-plugins-8f25d","openshift-network-diagnostics/network-check-target-7bk7n","openshift-cluster-node-tuning-operator/tuned-dljdk","openshift-dns/node-resolver-rkn2t","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-78.ec2.internal"]
Jul 07 11:26:55.816390 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.816363    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8f25d"
Jul 07 11:26:55.820093 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.820058    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 07 11:26:55.820363 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.820346    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 07 11:26:55.820637 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.820620    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rkn2t"
Jul 07 11:26:55.821877 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.821851    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 07 11:26:55.822065 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.822043    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-9sg47\""
Jul 07 11:26:55.822065 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.822026    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 07 11:26:55.822244 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.822225    2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 07 11:26:55.822888 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.822688    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 07 11:26:55.823002 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.822888    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vdrl5"
Jul 07 11:26:55.824529 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.824509    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-mpjj2\""
Jul 07 11:26:55.824699 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.824679    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 07 11:26:55.824932 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.824915    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 07 11:26:55.826123 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.826097    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.827685 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.827462    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-vdczj\""
Jul 07 11:26:55.827685 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.827486    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 07 11:26:55.827685 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.827502    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 07 11:26:55.827685 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.827578    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 07 11:26:55.828672 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.828649    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8f5jf"
Jul 07 11:26:55.828774 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:55.828736    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8f5jf" podUID="afdb1ba0-560a-4b0f-883b-aa960c6dd304"
Jul 07 11:26:55.828845 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.828804    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 07 11:26:55.829028 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.829006    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 07 11:26:55.829106 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.829066    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-cc9q6\""
Jul 07 11:26:55.829242 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.829224    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 07 11:26:55.830714 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.830692    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 07 11:26:55.830848 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.830747    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 07 11:26:55.830947 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.830779    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 07 11:26:55.831445 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.831144    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7tk6n"
Jul 07 11:26:55.833655 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.833633    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qh2dm"
Jul 07 11:26:55.834738 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.834720    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 07 11:26:55.834937 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.834760    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 07 11:26:55.835038 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.834813    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-8plfh\""
Jul 07 11:26:55.836324 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.836174    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-node-log\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.836324 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.836213    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-env-overrides\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.836324 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.836267    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-ovnkube-script-lib\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.836324 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.836299    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b9392160-8536-4d5e-916d-e2a0164db043-iptables-alerter-script\") pod \"iptables-alerter-vdrl5\" (UID: \"b9392160-8536-4d5e-916d-e2a0164db043\") " pod="openshift-network-operator/iptables-alerter-vdrl5"
Jul 07 11:26:55.836571 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.836328    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b9392160-8536-4d5e-916d-e2a0164db043-host-slash\") pod \"iptables-alerter-vdrl5\" (UID: \"b9392160-8536-4d5e-916d-e2a0164db043\") " pod="openshift-network-operator/iptables-alerter-vdrl5"
Jul 07 11:26:55.836571 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.836352    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-run-openvswitch\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.836571 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.836377    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/650c2a15-194e-47f4-99c8-9f94802d1e9b-tmp-dir\") pod \"node-resolver-rkn2t\" (UID: \"650c2a15-194e-47f4-99c8-9f94802d1e9b\") " pod="openshift-dns/node-resolver-rkn2t"
Jul 07 11:26:55.836571 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.836403    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-host-run-netns\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.836571 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.836431    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-host-cni-bin\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.836571 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.836459    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-host-cni-netd\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.836571 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.836486    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-ovn-node-metrics-cert\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.836571 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.836512    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/20c2dc90-b7d3-4add-af1f-31f1ba991c54-system-cni-dir\") pod \"multus-additional-cni-plugins-8f25d\" (UID: \"20c2dc90-b7d3-4add-af1f-31f1ba991c54\") " pod="openshift-multus/multus-additional-cni-plugins-8f25d"
Jul 07 11:26:55.836571 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.836541    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkt7z\" (UniqueName: \"kubernetes.io/projected/b9392160-8536-4d5e-916d-e2a0164db043-kube-api-access-zkt7z\") pod \"iptables-alerter-vdrl5\" (UID: \"b9392160-8536-4d5e-916d-e2a0164db043\") " pod="openshift-network-operator/iptables-alerter-vdrl5"
Jul 07 11:26:55.836571 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.836566    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-host-run-ovn-kubernetes\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.836973 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.836591    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/20c2dc90-b7d3-4add-af1f-31f1ba991c54-cnibin\") pod \"multus-additional-cni-plugins-8f25d\" (UID: \"20c2dc90-b7d3-4add-af1f-31f1ba991c54\") " pod="openshift-multus/multus-additional-cni-plugins-8f25d"
Jul 07 11:26:55.836973 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.836619    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-etc-openvswitch\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.836973 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.836665    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trlwh\" (UniqueName: \"kubernetes.io/projected/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-kube-api-access-trlwh\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.836973 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.836695    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/20c2dc90-b7d3-4add-af1f-31f1ba991c54-cni-binary-copy\") pod \"multus-additional-cni-plugins-8f25d\" (UID: \"20c2dc90-b7d3-4add-af1f-31f1ba991c54\") " pod="openshift-multus/multus-additional-cni-plugins-8f25d"
Jul 07 11:26:55.836973 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.836730    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/20c2dc90-b7d3-4add-af1f-31f1ba991c54-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8f25d\" (UID: \"20c2dc90-b7d3-4add-af1f-31f1ba991c54\") " pod="openshift-multus/multus-additional-cni-plugins-8f25d"
Jul 07 11:26:55.836973 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.836753    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/650c2a15-194e-47f4-99c8-9f94802d1e9b-hosts-file\") pod \"node-resolver-rkn2t\" (UID: \"650c2a15-194e-47f4-99c8-9f94802d1e9b\") " pod="openshift-dns/node-resolver-rkn2t"
Jul 07 11:26:55.836973 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.836783    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84slm\" (UniqueName: \"kubernetes.io/projected/650c2a15-194e-47f4-99c8-9f94802d1e9b-kube-api-access-84slm\") pod \"node-resolver-rkn2t\" (UID: \"650c2a15-194e-47f4-99c8-9f94802d1e9b\") " pod="openshift-dns/node-resolver-rkn2t"
Jul 07 11:26:55.836973 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.836821    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-log-socket\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.836973 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.836847    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-ovnkube-config\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.836973 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.836871    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/20c2dc90-b7d3-4add-af1f-31f1ba991c54-os-release\") pod \"multus-additional-cni-plugins-8f25d\" (UID: \"20c2dc90-b7d3-4add-af1f-31f1ba991c54\") " pod="openshift-multus/multus-additional-cni-plugins-8f25d"
Jul 07 11:26:55.836973 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.836897    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hfwt\" (UniqueName: \"kubernetes.io/projected/20c2dc90-b7d3-4add-af1f-31f1ba991c54-kube-api-access-8hfwt\") pod \"multus-additional-cni-plugins-8f25d\" (UID: \"20c2dc90-b7d3-4add-af1f-31f1ba991c54\") " pod="openshift-multus/multus-additional-cni-plugins-8f25d"
Jul 07 11:26:55.836973 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.836944    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-var-lib-openvswitch\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.836973 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.836971    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.837510 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.837005    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-host-kubelet\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.837510 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.837034    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-systemd-units\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.837510 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.837059    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-run-ovn\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.837510 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.837085    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/20c2dc90-b7d3-4add-af1f-31f1ba991c54-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8f25d\" (UID: \"20c2dc90-b7d3-4add-af1f-31f1ba991c54\") " pod="openshift-multus/multus-additional-cni-plugins-8f25d"
Jul 07 11:26:55.837510 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.837133    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/20c2dc90-b7d3-4add-af1f-31f1ba991c54-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8f25d\" (UID: \"20c2dc90-b7d3-4add-af1f-31f1ba991c54\") " pod="openshift-multus/multus-additional-cni-plugins-8f25d"
Jul 07 11:26:55.837510 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.837159    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-host-slash\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.837510 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.837189    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-run-systemd\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.837510 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.837362    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 07 11:26:55.837952 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.837535    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lmxhq"
Jul 07 11:26:55.837952 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.837644    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-bszwz\""
Jul 07 11:26:55.837952 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.837761    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 07 11:26:55.837952 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.837853    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 07 11:26:55.838545 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.838526    2570 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 07 11:26:55.839584 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.839395    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ppmbz"
Jul 07 11:26:55.839584 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:55.839501    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ppmbz" podUID="f75e11d0-c0de-4a8d-849d-807a87aa1497"
Jul 07 11:26:55.840043 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.840015    2570 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 11:26:55.841829 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.841809    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 07 11:26:55.841930 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.841811    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 07 11:26:55.842154 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.842135    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-bnsx4\""
Jul 07 11:26:55.842648 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.842523    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 07 11:26:55.842878 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.842861    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lj4rs"
Jul 07 11:26:55.845411 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.845378    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7bk7n"
Jul 07 11:26:55.845547 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:55.845500    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7bk7n" podUID="732e808f-49d9-44a1-b93f-1cdb95c31e0b"
Jul 07 11:26:55.845768 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.845749    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 07 11:26:55.845928 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.845910    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-twsnl\""
Jul 07 11:26:55.848114 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.847780    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dljdk"
Jul 07 11:26:55.850978 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.850691    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 07 11:26:55.850978 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.850797    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-jcrvv\""
Jul 07 11:26:55.850978 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.850700    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 07 11:26:55.864807 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.864775    2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-hcmz6"
Jul 07 11:26:55.873061 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.872630    2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-hcmz6"
Jul 07 11:26:55.925411 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.924153    2570 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 07 11:26:55.937428 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.937395    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e54ef86d-1806-4e84-a235-aba912f88f58-serviceca\") pod \"node-ca-qh2dm\" (UID: \"e54ef86d-1806-4e84-a235-aba912f88f58\") " pod="openshift-image-registry/node-ca-qh2dm"
Jul 07 11:26:55.937581 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.937439    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-host-kubelet\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.937581 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.937470    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-systemd-units\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.937581 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.937496    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/20c2dc90-b7d3-4add-af1f-31f1ba991c54-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8f25d\" (UID: \"20c2dc90-b7d3-4add-af1f-31f1ba991c54\") " pod="openshift-multus/multus-additional-cni-plugins-8f25d"
Jul 07 11:26:55.937581 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.937519    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-host-kubelet\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.937581 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.937522    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/20c2dc90-b7d3-4add-af1f-31f1ba991c54-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8f25d\" (UID: \"20c2dc90-b7d3-4add-af1f-31f1ba991c54\") " pod="openshift-multus/multus-additional-cni-plugins-8f25d"
Jul 07 11:26:55.937581 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.937554    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4e27418c-5ace-4a75-a4b9-931fffa64650-multus-cni-dir\") pod \"multus-lj4rs\" (UID: \"4e27418c-5ace-4a75-a4b9-931fffa64650\") " pod="openshift-multus/multus-lj4rs"
Jul 07 11:26:55.937869 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.937584    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-run-systemd\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.937869 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.937601    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-systemd-units\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.937869 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.937612    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-env-overrides\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.937869 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.937638    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-ovnkube-script-lib\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.937869 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.937665    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8chf\" (UniqueName: \"kubernetes.io/projected/f75e11d0-c0de-4a8d-849d-807a87aa1497-kube-api-access-w8chf\") pod \"network-metrics-daemon-ppmbz\" (UID: \"f75e11d0-c0de-4a8d-849d-807a87aa1497\") " pod="openshift-multus/network-metrics-daemon-ppmbz"
Jul 07 11:26:55.937869 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.937689    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b2009767-f15b-464f-8a3f-87335106fe11-var-lib-kubelet\") pod \"tuned-dljdk\" (UID: \"b2009767-f15b-464f-8a3f-87335106fe11\") " pod="openshift-cluster-node-tuning-operator/tuned-dljdk"
Jul 07 11:26:55.937869 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.937714    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ea85da56-a626-4fc7-a0d4-31b668987496-socket-dir\") pod \"aws-ebs-csi-driver-node-lmxhq\" (UID: \"ea85da56-a626-4fc7-a0d4-31b668987496\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lmxhq"
Jul 07 11:26:55.937869 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.937740    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b9392160-8536-4d5e-916d-e2a0164db043-iptables-alerter-script\") pod \"iptables-alerter-vdrl5\" (UID: \"b9392160-8536-4d5e-916d-e2a0164db043\") " pod="openshift-network-operator/iptables-alerter-vdrl5"
Jul 07 11:26:55.937869 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.937766    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-run-openvswitch\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.937869 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.937789    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/20c2dc90-b7d3-4add-af1f-31f1ba991c54-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8f25d\" (UID: \"20c2dc90-b7d3-4add-af1f-31f1ba991c54\") " pod="openshift-multus/multus-additional-cni-plugins-8f25d"
Jul 07 11:26:55.937869 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.937791    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/650c2a15-194e-47f4-99c8-9f94802d1e9b-tmp-dir\") pod \"node-resolver-rkn2t\" (UID: \"650c2a15-194e-47f4-99c8-9f94802d1e9b\") " pod="openshift-dns/node-resolver-rkn2t"
Jul 07 11:26:55.937869 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.937819    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/afdb1ba0-560a-4b0f-883b-aa960c6dd304-original-pull-secret\") pod \"global-pull-secret-syncer-8f5jf\" (UID: \"afdb1ba0-560a-4b0f-883b-aa960c6dd304\") " pod="kube-system/global-pull-secret-syncer-8f5jf"
Jul 07 11:26:55.937869 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.937842    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/af613b4a-9c2e-4b9b-8a10-0ca67f1f9159-agent-certs\") pod \"konnectivity-agent-7tk6n\" (UID: \"af613b4a-9c2e-4b9b-8a10-0ca67f1f9159\") " pod="kube-system/konnectivity-agent-7tk6n"
Jul 07 11:26:55.937869 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.937868    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hb26c\" (UniqueName: \"kubernetes.io/projected/732e808f-49d9-44a1-b93f-1cdb95c31e0b-kube-api-access-hb26c\") pod \"network-check-target-7bk7n\" (UID: \"732e808f-49d9-44a1-b93f-1cdb95c31e0b\") " pod="openshift-network-diagnostics/network-check-target-7bk7n"
Jul 07 11:26:55.938554 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.937894    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b2009767-f15b-464f-8a3f-87335106fe11-etc-sysconfig\") pod \"tuned-dljdk\" (UID: \"b2009767-f15b-464f-8a3f-87335106fe11\") " pod="openshift-cluster-node-tuning-operator/tuned-dljdk"
Jul 07 11:26:55.938554 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.937921    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b2009767-f15b-464f-8a3f-87335106fe11-sys\") pod \"tuned-dljdk\" (UID: \"b2009767-f15b-464f-8a3f-87335106fe11\") " pod="openshift-cluster-node-tuning-operator/tuned-dljdk"
Jul 07 11:26:55.938554 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.937946    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-host-cni-bin\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.938554 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.937973    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-ovn-node-metrics-cert\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.938554 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.937996    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b2009767-f15b-464f-8a3f-87335106fe11-etc-tuned\") pod \"tuned-dljdk\" (UID: \"b2009767-f15b-464f-8a3f-87335106fe11\") " pod="openshift-cluster-node-tuning-operator/tuned-dljdk"
Jul 07 11:26:55.938554 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.938019    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b2009767-f15b-464f-8a3f-87335106fe11-tmp\") pod \"tuned-dljdk\" (UID: \"b2009767-f15b-464f-8a3f-87335106fe11\") " pod="openshift-cluster-node-tuning-operator/tuned-dljdk"
Jul 07 11:26:55.938554 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.938053    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4e27418c-5ace-4a75-a4b9-931fffa64650-cni-binary-copy\") pod \"multus-lj4rs\" (UID: \"4e27418c-5ace-4a75-a4b9-931fffa64650\") " pod="openshift-multus/multus-lj4rs"
Jul 07 11:26:55.938554 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.938081    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4e27418c-5ace-4a75-a4b9-931fffa64650-host-run-k8s-cni-cncf-io\") pod \"multus-lj4rs\" (UID: \"4e27418c-5ace-4a75-a4b9-931fffa64650\") " pod="openshift-multus/multus-lj4rs"
Jul 07 11:26:55.938554 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.938107    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e27418c-5ace-4a75-a4b9-931fffa64650-host-var-lib-kubelet\") pod \"multus-lj4rs\" (UID: \"4e27418c-5ace-4a75-a4b9-931fffa64650\") " pod="openshift-multus/multus-lj4rs"
Jul 07 11:26:55.938554 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.938135    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkt7z\" (UniqueName: \"kubernetes.io/projected/b9392160-8536-4d5e-916d-e2a0164db043-kube-api-access-zkt7z\") pod \"iptables-alerter-vdrl5\" (UID: \"b9392160-8536-4d5e-916d-e2a0164db043\") " pod="openshift-network-operator/iptables-alerter-vdrl5"
Jul 07 11:26:55.938554 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.938161    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/20c2dc90-b7d3-4add-af1f-31f1ba991c54-cnibin\") pod \"multus-additional-cni-plugins-8f25d\" (UID: \"20c2dc90-b7d3-4add-af1f-31f1ba991c54\") " pod="openshift-multus/multus-additional-cni-plugins-8f25d"
Jul 07 11:26:55.938554 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.938191    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttqsw\" (UniqueName: \"kubernetes.io/projected/ea85da56-a626-4fc7-a0d4-31b668987496-kube-api-access-ttqsw\") pod \"aws-ebs-csi-driver-node-lmxhq\" (UID: \"ea85da56-a626-4fc7-a0d4-31b668987496\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lmxhq"
Jul 07 11:26:55.938554 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.938229    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4e27418c-5ace-4a75-a4b9-931fffa64650-os-release\") pod \"multus-lj4rs\" (UID: \"4e27418c-5ace-4a75-a4b9-931fffa64650\") " pod="openshift-multus/multus-lj4rs"
Jul 07 11:26:55.938554 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.938262    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/20c2dc90-b7d3-4add-af1f-31f1ba991c54-cni-binary-copy\") pod \"multus-additional-cni-plugins-8f25d\" (UID: \"20c2dc90-b7d3-4add-af1f-31f1ba991c54\") " pod="openshift-multus/multus-additional-cni-plugins-8f25d"
Jul 07 11:26:55.938554 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.938322    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/20c2dc90-b7d3-4add-af1f-31f1ba991c54-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8f25d\" (UID: \"20c2dc90-b7d3-4add-af1f-31f1ba991c54\") " pod="openshift-multus/multus-additional-cni-plugins-8f25d"
Jul 07 11:26:55.938554 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.938345    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/650c2a15-194e-47f4-99c8-9f94802d1e9b-hosts-file\") pod \"node-resolver-rkn2t\" (UID: \"650c2a15-194e-47f4-99c8-9f94802d1e9b\") " pod="openshift-dns/node-resolver-rkn2t"
Jul 07 11:26:55.938554 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.938369    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84slm\" (UniqueName: \"kubernetes.io/projected/650c2a15-194e-47f4-99c8-9f94802d1e9b-kube-api-access-84slm\") pod \"node-resolver-rkn2t\" (UID: \"650c2a15-194e-47f4-99c8-9f94802d1e9b\") " pod="openshift-dns/node-resolver-rkn2t"
Jul 07 11:26:55.939449 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.938396    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/afdb1ba0-560a-4b0f-883b-aa960c6dd304-dbus\") pod \"global-pull-secret-syncer-8f5jf\" (UID: \"afdb1ba0-560a-4b0f-883b-aa960c6dd304\") " pod="kube-system/global-pull-secret-syncer-8f5jf"
Jul 07 11:26:55.939449 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.938423    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khf5x\" (UniqueName: \"kubernetes.io/projected/e54ef86d-1806-4e84-a235-aba912f88f58-kube-api-access-khf5x\") pod \"node-ca-qh2dm\" (UID: \"e54ef86d-1806-4e84-a235-aba912f88f58\") " pod="openshift-image-registry/node-ca-qh2dm"
Jul 07 11:26:55.939449 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.938454    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-log-socket\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.939449 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.938464    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-ovnkube-script-lib\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.939449 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.938480    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/20c2dc90-b7d3-4add-af1f-31f1ba991c54-os-release\") pod \"multus-additional-cni-plugins-8f25d\" (UID: \"20c2dc90-b7d3-4add-af1f-31f1ba991c54\") " pod="openshift-multus/multus-additional-cni-plugins-8f25d"
Jul 07 11:26:55.939449 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.938506    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hfwt\" (UniqueName: \"kubernetes.io/projected/20c2dc90-b7d3-4add-af1f-31f1ba991c54-kube-api-access-8hfwt\") pod \"multus-additional-cni-plugins-8f25d\" (UID: \"20c2dc90-b7d3-4add-af1f-31f1ba991c54\") " pod="openshift-multus/multus-additional-cni-plugins-8f25d"
Jul 07 11:26:55.939449 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.938531    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b2009767-f15b-464f-8a3f-87335106fe11-run\") pod \"tuned-dljdk\" (UID: \"b2009767-f15b-464f-8a3f-87335106fe11\") " pod="openshift-cluster-node-tuning-operator/tuned-dljdk"
Jul 07 11:26:55.939449 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.938558    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ea85da56-a626-4fc7-a0d4-31b668987496-sys-fs\") pod \"aws-ebs-csi-driver-node-lmxhq\" (UID: \"ea85da56-a626-4fc7-a0d4-31b668987496\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lmxhq"
Jul 07 11:26:55.939449 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.938583    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4e27418c-5ace-4a75-a4b9-931fffa64650-multus-conf-dir\") pod \"multus-lj4rs\" (UID: \"4e27418c-5ace-4a75-a4b9-931fffa64650\") " pod="openshift-multus/multus-lj4rs"
Jul 07 11:26:55.939449 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.938617    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-var-lib-openvswitch\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.939449 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.938649    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.939449 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.938677    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4e27418c-5ace-4a75-a4b9-931fffa64650-host-var-lib-cni-multus\") pod \"multus-lj4rs\" (UID: \"4e27418c-5ace-4a75-a4b9-931fffa64650\") " pod="openshift-multus/multus-lj4rs"
Jul 07 11:26:55.939449 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.938703    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4e27418c-5ace-4a75-a4b9-931fffa64650-multus-daemon-config\") pod \"multus-lj4rs\" (UID: \"4e27418c-5ace-4a75-a4b9-931fffa64650\") " pod="openshift-multus/multus-lj4rs"
Jul 07 11:26:55.939449 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.938745    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-run-ovn\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.939449 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.938773    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f75e11d0-c0de-4a8d-849d-807a87aa1497-metrics-certs\") pod \"network-metrics-daemon-ppmbz\" (UID: \"f75e11d0-c0de-4a8d-849d-807a87aa1497\") " pod="openshift-multus/network-metrics-daemon-ppmbz"
Jul 07 11:26:55.939449 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.938800    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ea85da56-a626-4fc7-a0d4-31b668987496-etc-selinux\") pod \"aws-ebs-csi-driver-node-lmxhq\" (UID: \"ea85da56-a626-4fc7-a0d4-31b668987496\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lmxhq"
Jul 07 11:26:55.939449 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.938830    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4e27418c-5ace-4a75-a4b9-931fffa64650-system-cni-dir\") pod \"multus-lj4rs\" (UID: \"4e27418c-5ace-4a75-a4b9-931fffa64650\") " pod="openshift-multus/multus-lj4rs"
Jul 07 11:26:55.940324 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.938857    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4e27418c-5ace-4a75-a4b9-931fffa64650-cnibin\") pod \"multus-lj4rs\" (UID: \"4e27418c-5ace-4a75-a4b9-931fffa64650\") " pod="openshift-multus/multus-lj4rs"
Jul 07 11:26:55.940324 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.938889    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-host-slash\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.940324 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.938917    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-node-log\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.940324 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.938943    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b2009767-f15b-464f-8a3f-87335106fe11-etc-sysctl-d\") pod \"tuned-dljdk\" (UID: \"b2009767-f15b-464f-8a3f-87335106fe11\") " pod="openshift-cluster-node-tuning-operator/tuned-dljdk"
Jul 07 11:26:55.940324 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.938969    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b2009767-f15b-464f-8a3f-87335106fe11-etc-systemd\") pod \"tuned-dljdk\" (UID: \"b2009767-f15b-464f-8a3f-87335106fe11\") " pod="openshift-cluster-node-tuning-operator/tuned-dljdk"
Jul 07 11:26:55.940324 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.938983    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/20c2dc90-b7d3-4add-af1f-31f1ba991c54-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8f25d\" (UID: \"20c2dc90-b7d3-4add-af1f-31f1ba991c54\") " pod="openshift-multus/multus-additional-cni-plugins-8f25d"
Jul 07 11:26:55.940324 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.938997    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4e27418c-5ace-4a75-a4b9-931fffa64650-multus-socket-dir-parent\") pod \"multus-lj4rs\" (UID: \"4e27418c-5ace-4a75-a4b9-931fffa64650\") " pod="openshift-multus/multus-lj4rs"
Jul 07 11:26:55.940324 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.939041    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b9392160-8536-4d5e-916d-e2a0164db043-host-slash\") pod \"iptables-alerter-vdrl5\" (UID: \"b9392160-8536-4d5e-916d-e2a0164db043\") " pod="openshift-network-operator/iptables-alerter-vdrl5"
Jul 07 11:26:55.940324 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.939067    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b2009767-f15b-464f-8a3f-87335106fe11-lib-modules\") pod \"tuned-dljdk\" (UID: \"b2009767-f15b-464f-8a3f-87335106fe11\") " pod="openshift-cluster-node-tuning-operator/tuned-dljdk"
Jul 07 11:26:55.940324 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.939096    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e54ef86d-1806-4e84-a235-aba912f88f58-host\") pod \"node-ca-qh2dm\" (UID: \"e54ef86d-1806-4e84-a235-aba912f88f58\") " pod="openshift-image-registry/node-ca-qh2dm"
Jul 07 11:26:55.940324 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.939128    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4e27418c-5ace-4a75-a4b9-931fffa64650-hostroot\") pod \"multus-lj4rs\" (UID: \"4e27418c-5ace-4a75-a4b9-931fffa64650\") " pod="openshift-multus/multus-lj4rs"
Jul 07 11:26:55.940324 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.939157    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4e27418c-5ace-4a75-a4b9-931fffa64650-etc-kubernetes\") pod \"multus-lj4rs\" (UID: \"4e27418c-5ace-4a75-a4b9-931fffa64650\") " pod="openshift-multus/multus-lj4rs"
Jul 07 11:26:55.940324 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.939189    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-host-run-netns\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.940324 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.939216    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-host-cni-netd\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.940324 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.939246    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/20c2dc90-b7d3-4add-af1f-31f1ba991c54-system-cni-dir\") pod \"multus-additional-cni-plugins-8f25d\" (UID: \"20c2dc90-b7d3-4add-af1f-31f1ba991c54\") " pod="openshift-multus/multus-additional-cni-plugins-8f25d"
Jul 07 11:26:55.940324 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.939275    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b2009767-f15b-464f-8a3f-87335106fe11-host\") pod \"tuned-dljdk\" (UID: \"b2009767-f15b-464f-8a3f-87335106fe11\") " pod="openshift-cluster-node-tuning-operator/tuned-dljdk"
Jul 07 11:26:55.940324 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.939320    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4e27418c-5ace-4a75-a4b9-931fffa64650-host-run-netns\") pod \"multus-lj4rs\" (UID: \"4e27418c-5ace-4a75-a4b9-931fffa64650\") " pod="openshift-multus/multus-lj4rs"
Jul 07 11:26:55.941176 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.939352    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-host-run-ovn-kubernetes\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.941176 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.939367    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-env-overrides\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.941176 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.939380    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/afdb1ba0-560a-4b0f-883b-aa960c6dd304-kubelet-config\") pod \"global-pull-secret-syncer-8f5jf\" (UID: \"afdb1ba0-560a-4b0f-883b-aa960c6dd304\") " pod="kube-system/global-pull-secret-syncer-8f5jf"
Jul 07 11:26:55.941176 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.939406    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b2009767-f15b-464f-8a3f-87335106fe11-etc-modprobe-d\") pod \"tuned-dljdk\" (UID: \"b2009767-f15b-464f-8a3f-87335106fe11\") " pod="openshift-cluster-node-tuning-operator/tuned-dljdk"
Jul 07 11:26:55.941176 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.939431    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b2009767-f15b-464f-8a3f-87335106fe11-etc-kubernetes\") pod \"tuned-dljdk\" (UID: \"b2009767-f15b-464f-8a3f-87335106fe11\") " pod="openshift-cluster-node-tuning-operator/tuned-dljdk"
Jul 07 11:26:55.941176 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.939456    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b2009767-f15b-464f-8a3f-87335106fe11-etc-sysctl-conf\") pod \"tuned-dljdk\" (UID: \"b2009767-f15b-464f-8a3f-87335106fe11\") " pod="openshift-cluster-node-tuning-operator/tuned-dljdk"
Jul 07 11:26:55.941176 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.939481    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpvn2\" (UniqueName: \"kubernetes.io/projected/b2009767-f15b-464f-8a3f-87335106fe11-kube-api-access-mpvn2\") pod \"tuned-dljdk\" (UID: \"b2009767-f15b-464f-8a3f-87335106fe11\") " pod="openshift-cluster-node-tuning-operator/tuned-dljdk"
Jul 07 11:26:55.941176 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.939509    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ea85da56-a626-4fc7-a0d4-31b668987496-device-dir\") pod \"aws-ebs-csi-driver-node-lmxhq\" (UID: \"ea85da56-a626-4fc7-a0d4-31b668987496\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lmxhq"
Jul 07 11:26:55.941176 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.939540    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-etc-openvswitch\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.941176 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.939540    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-78.ec2.internal" event={"ID":"4f2009fc30615ef1da5c98996e493d3f","Type":"ContainerStarted","Data":"9a943cb5da943f857849307cb48ff32760b49ce839664d261d1bc419cb3584fa"}
Jul 07 11:26:55.941176 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.939572    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trlwh\" (UniqueName: \"kubernetes.io/projected/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-kube-api-access-trlwh\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.941176 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.939642    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ea85da56-a626-4fc7-a0d4-31b668987496-registration-dir\") pod \"aws-ebs-csi-driver-node-lmxhq\" (UID: \"ea85da56-a626-4fc7-a0d4-31b668987496\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lmxhq"
Jul 07 11:26:55.941176 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.939690    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-ovnkube-config\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.941176 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.939719    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ea85da56-a626-4fc7-a0d4-31b668987496-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lmxhq\" (UID: \"ea85da56-a626-4fc7-a0d4-31b668987496\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lmxhq"
Jul 07 11:26:55.941176 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.939747    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4e27418c-5ace-4a75-a4b9-931fffa64650-host-var-lib-cni-bin\") pod \"multus-lj4rs\" (UID: \"4e27418c-5ace-4a75-a4b9-931fffa64650\") " pod="openshift-multus/multus-lj4rs"
Jul 07 11:26:55.941176 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.939773    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4e27418c-5ace-4a75-a4b9-931fffa64650-host-run-multus-certs\") pod \"multus-lj4rs\" (UID: \"4e27418c-5ace-4a75-a4b9-931fffa64650\") " pod="openshift-multus/multus-lj4rs"
Jul 07 11:26:55.941176 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.939799    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xv7jm\" (UniqueName: \"kubernetes.io/projected/4e27418c-5ace-4a75-a4b9-931fffa64650-kube-api-access-xv7jm\") pod \"multus-lj4rs\" (UID: \"4e27418c-5ace-4a75-a4b9-931fffa64650\") " pod="openshift-multus/multus-lj4rs"
Jul 07 11:26:55.942021 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.939825    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/af613b4a-9c2e-4b9b-8a10-0ca67f1f9159-konnectivity-ca\") pod \"konnectivity-agent-7tk6n\" (UID: \"af613b4a-9c2e-4b9b-8a10-0ca67f1f9159\") " pod="kube-system/konnectivity-agent-7tk6n"
Jul 07 11:26:55.942021 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.940005    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-run-systemd\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.942021 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.940123    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-etc-openvswitch\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.942021 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.940144    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-var-lib-openvswitch\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.942021 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.940181    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-run-ovn\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.942021 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.940202    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b9392160-8536-4d5e-916d-e2a0164db043-host-slash\") pod \"iptables-alerter-vdrl5\" (UID: \"b9392160-8536-4d5e-916d-e2a0164db043\") " pod="openshift-network-operator/iptables-alerter-vdrl5"
Jul 07 11:26:55.942021 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.940222    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/20c2dc90-b7d3-4add-af1f-31f1ba991c54-cnibin\") pod \"multus-additional-cni-plugins-8f25d\" (UID: \"20c2dc90-b7d3-4add-af1f-31f1ba991c54\") " pod="openshift-multus/multus-additional-cni-plugins-8f25d"
Jul 07 11:26:55.942021 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.940277    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.942021 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.940292    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/20c2dc90-b7d3-4add-af1f-31f1ba991c54-os-release\") pod \"multus-additional-cni-plugins-8f25d\" (UID: \"20c2dc90-b7d3-4add-af1f-31f1ba991c54\") " pod="openshift-multus/multus-additional-cni-plugins-8f25d"
Jul 07 11:26:55.942021 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.940318    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-run-openvswitch\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.942021 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.940323    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-host-run-netns\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.942021 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.940378    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-host-run-ovn-kubernetes\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.942021 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.940421    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-host-cni-netd\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.942021 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.940459    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/20c2dc90-b7d3-4add-af1f-31f1ba991c54-system-cni-dir\") pod \"multus-additional-cni-plugins-8f25d\" (UID: \"20c2dc90-b7d3-4add-af1f-31f1ba991c54\") " pod="openshift-multus/multus-additional-cni-plugins-8f25d"
Jul 07 11:26:55.942021 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.940497    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-host-slash\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.942021 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.940564    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-node-log\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.942021 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.940603    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-log-socket\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.942853 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.940632    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/650c2a15-194e-47f4-99c8-9f94802d1e9b-tmp-dir\") pod \"node-resolver-rkn2t\" (UID: \"650c2a15-194e-47f4-99c8-9f94802d1e9b\") " pod="openshift-dns/node-resolver-rkn2t"
Jul 07 11:26:55.942853 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.940647    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-host-cni-bin\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.942853 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.940635    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b9392160-8536-4d5e-916d-e2a0164db043-iptables-alerter-script\") pod \"iptables-alerter-vdrl5\" (UID: \"b9392160-8536-4d5e-916d-e2a0164db043\") " pod="openshift-network-operator/iptables-alerter-vdrl5"
Jul 07 11:26:55.942853 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.940708    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/650c2a15-194e-47f4-99c8-9f94802d1e9b-hosts-file\") pod \"node-resolver-rkn2t\" (UID: \"650c2a15-194e-47f4-99c8-9f94802d1e9b\") " pod="openshift-dns/node-resolver-rkn2t"
Jul 07 11:26:55.942853 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.940835    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-ovnkube-config\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.942853 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.941032    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/20c2dc90-b7d3-4add-af1f-31f1ba991c54-cni-binary-copy\") pod \"multus-additional-cni-plugins-8f25d\" (UID: \"20c2dc90-b7d3-4add-af1f-31f1ba991c54\") " pod="openshift-multus/multus-additional-cni-plugins-8f25d"
Jul 07 11:26:55.942853 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.941042    2570 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 07 11:26:55.942853 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.941286    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/20c2dc90-b7d3-4add-af1f-31f1ba991c54-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8f25d\" (UID: \"20c2dc90-b7d3-4add-af1f-31f1ba991c54\") " pod="openshift-multus/multus-additional-cni-plugins-8f25d"
Jul 07 11:26:55.943258 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.943040    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-78.ec2.internal" event={"ID":"c9aa93f97f9dd2447825488095d15a9f","Type":"ContainerStarted","Data":"db88debb254d9d89eaed6846daf44b32af3680a893dab8f7d6f9c35c5f571cf2"}
Jul 07 11:26:55.944757 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.944730    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-ovn-node-metrics-cert\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.949857 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.949827    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkt7z\" (UniqueName: \"kubernetes.io/projected/b9392160-8536-4d5e-916d-e2a0164db043-kube-api-access-zkt7z\") pod \"iptables-alerter-vdrl5\" (UID: \"b9392160-8536-4d5e-916d-e2a0164db043\") " pod="openshift-network-operator/iptables-alerter-vdrl5"
Jul 07 11:26:55.950870 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.950655    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-trlwh\" (UniqueName: \"kubernetes.io/projected/6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b-kube-api-access-trlwh\") pod \"ovnkube-node-pm4tw\" (UID: \"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b\") " pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:55.950990 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.950968    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-84slm\" (UniqueName: \"kubernetes.io/projected/650c2a15-194e-47f4-99c8-9f94802d1e9b-kube-api-access-84slm\") pod \"node-resolver-rkn2t\" (UID: \"650c2a15-194e-47f4-99c8-9f94802d1e9b\") " pod="openshift-dns/node-resolver-rkn2t"
Jul 07 11:26:55.951570 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:55.951546    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hfwt\" (UniqueName: \"kubernetes.io/projected/20c2dc90-b7d3-4add-af1f-31f1ba991c54-kube-api-access-8hfwt\") pod \"multus-additional-cni-plugins-8f25d\" (UID: \"20c2dc90-b7d3-4add-af1f-31f1ba991c54\") " pod="openshift-multus/multus-additional-cni-plugins-8f25d"
Jul 07 11:26:56.040700 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.040660    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b2009767-f15b-464f-8a3f-87335106fe11-etc-tuned\") pod \"tuned-dljdk\" (UID: \"b2009767-f15b-464f-8a3f-87335106fe11\") " pod="openshift-cluster-node-tuning-operator/tuned-dljdk"
Jul 07 11:26:56.040888 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.040715    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b2009767-f15b-464f-8a3f-87335106fe11-tmp\") pod \"tuned-dljdk\" (UID: \"b2009767-f15b-464f-8a3f-87335106fe11\") " pod="openshift-cluster-node-tuning-operator/tuned-dljdk"
Jul 07 11:26:56.040888 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.040750    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4e27418c-5ace-4a75-a4b9-931fffa64650-cni-binary-copy\") pod \"multus-lj4rs\" (UID: \"4e27418c-5ace-4a75-a4b9-931fffa64650\") " pod="openshift-multus/multus-lj4rs"
Jul 07 11:26:56.040888 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.040786    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4e27418c-5ace-4a75-a4b9-931fffa64650-host-run-k8s-cni-cncf-io\") pod \"multus-lj4rs\" (UID: \"4e27418c-5ace-4a75-a4b9-931fffa64650\") " pod="openshift-multus/multus-lj4rs"
Jul 07 11:26:56.040888 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.040818    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e27418c-5ace-4a75-a4b9-931fffa64650-host-var-lib-kubelet\") pod \"multus-lj4rs\" (UID: \"4e27418c-5ace-4a75-a4b9-931fffa64650\") " pod="openshift-multus/multus-lj4rs"
Jul 07 11:26:56.041112 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.040889    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttqsw\" (UniqueName: \"kubernetes.io/projected/ea85da56-a626-4fc7-a0d4-31b668987496-kube-api-access-ttqsw\") pod \"aws-ebs-csi-driver-node-lmxhq\" (UID: \"ea85da56-a626-4fc7-a0d4-31b668987496\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lmxhq"
Jul 07 11:26:56.041112 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.040921    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4e27418c-5ace-4a75-a4b9-931fffa64650-os-release\") pod \"multus-lj4rs\" (UID: \"4e27418c-5ace-4a75-a4b9-931fffa64650\") " pod="openshift-multus/multus-lj4rs"
Jul 07 11:26:56.041112 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.040954    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/afdb1ba0-560a-4b0f-883b-aa960c6dd304-dbus\") pod \"global-pull-secret-syncer-8f5jf\" (UID: \"afdb1ba0-560a-4b0f-883b-aa960c6dd304\") " pod="kube-system/global-pull-secret-syncer-8f5jf"
Jul 07 11:26:56.041112 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.040984    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khf5x\" (UniqueName: \"kubernetes.io/projected/e54ef86d-1806-4e84-a235-aba912f88f58-kube-api-access-khf5x\") pod \"node-ca-qh2dm\" (UID: \"e54ef86d-1806-4e84-a235-aba912f88f58\") " pod="openshift-image-registry/node-ca-qh2dm"
Jul 07 11:26:56.041112 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.041019    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b2009767-f15b-464f-8a3f-87335106fe11-run\") pod \"tuned-dljdk\" (UID: \"b2009767-f15b-464f-8a3f-87335106fe11\") " pod="openshift-cluster-node-tuning-operator/tuned-dljdk"
Jul 07 11:26:56.041112 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.041033    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4e27418c-5ace-4a75-a4b9-931fffa64650-host-run-k8s-cni-cncf-io\") pod \"multus-lj4rs\" (UID: \"4e27418c-5ace-4a75-a4b9-931fffa64650\") " pod="openshift-multus/multus-lj4rs"
Jul 07 11:26:56.041112 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.041043    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ea85da56-a626-4fc7-a0d4-31b668987496-sys-fs\") pod \"aws-ebs-csi-driver-node-lmxhq\" (UID: \"ea85da56-a626-4fc7-a0d4-31b668987496\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lmxhq"
Jul 07 11:26:56.041112 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.041086    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4e27418c-5ace-4a75-a4b9-931fffa64650-multus-conf-dir\") pod \"multus-lj4rs\" (UID: \"4e27418c-5ace-4a75-a4b9-931fffa64650\") " pod="openshift-multus/multus-lj4rs"
Jul 07 11:26:56.041112 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.041112    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ea85da56-a626-4fc7-a0d4-31b668987496-sys-fs\") pod \"aws-ebs-csi-driver-node-lmxhq\" (UID: \"ea85da56-a626-4fc7-a0d4-31b668987496\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lmxhq"
Jul 07 11:26:56.041578 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.041124    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4e27418c-5ace-4a75-a4b9-931fffa64650-host-var-lib-cni-multus\") pod \"multus-lj4rs\" (UID: \"4e27418c-5ace-4a75-a4b9-931fffa64650\") " pod="openshift-multus/multus-lj4rs"
Jul 07 11:26:56.041578 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.041187    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e27418c-5ace-4a75-a4b9-931fffa64650-host-var-lib-kubelet\") pod \"multus-lj4rs\" (UID: \"4e27418c-5ace-4a75-a4b9-931fffa64650\") " pod="openshift-multus/multus-lj4rs"
Jul 07 11:26:56.041578 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.041201    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4e27418c-5ace-4a75-a4b9-931fffa64650-multus-daemon-config\") pod \"multus-lj4rs\" (UID: \"4e27418c-5ace-4a75-a4b9-931fffa64650\") " pod="openshift-multus/multus-lj4rs"
Jul 07 11:26:56.041578 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.041238    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f75e11d0-c0de-4a8d-849d-807a87aa1497-metrics-certs\") pod \"network-metrics-daemon-ppmbz\" (UID: \"f75e11d0-c0de-4a8d-849d-807a87aa1497\") " pod="openshift-multus/network-metrics-daemon-ppmbz"
Jul 07 11:26:56.041578 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.041271    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ea85da56-a626-4fc7-a0d4-31b668987496-etc-selinux\") pod \"aws-ebs-csi-driver-node-lmxhq\" (UID: \"ea85da56-a626-4fc7-a0d4-31b668987496\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lmxhq"
Jul 07 11:26:56.041578 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.041295    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4e27418c-5ace-4a75-a4b9-931fffa64650-os-release\") pod \"multus-lj4rs\" (UID: \"4e27418c-5ace-4a75-a4b9-931fffa64650\") " pod="openshift-multus/multus-lj4rs"
Jul 07 11:26:56.041578 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.041318    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4e27418c-5ace-4a75-a4b9-931fffa64650-system-cni-dir\") pod \"multus-lj4rs\" (UID: \"4e27418c-5ace-4a75-a4b9-931fffa64650\") " pod="openshift-multus/multus-lj4rs"
Jul 07 11:26:56.041578 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.041349    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4e27418c-5ace-4a75-a4b9-931fffa64650-cnibin\") pod \"multus-lj4rs\" (UID: \"4e27418c-5ace-4a75-a4b9-931fffa64650\") " pod="openshift-multus/multus-lj4rs"
Jul 07 11:26:56.041578 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.041382    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b2009767-f15b-464f-8a3f-87335106fe11-etc-sysctl-d\") pod \"tuned-dljdk\" (UID: \"b2009767-f15b-464f-8a3f-87335106fe11\") " pod="openshift-cluster-node-tuning-operator/tuned-dljdk"
Jul 07 11:26:56.041578 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.041412    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b2009767-f15b-464f-8a3f-87335106fe11-etc-systemd\") pod \"tuned-dljdk\" (UID: \"b2009767-f15b-464f-8a3f-87335106fe11\") " pod="openshift-cluster-node-tuning-operator/tuned-dljdk"
Jul 07 11:26:56.041578 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.041432    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/afdb1ba0-560a-4b0f-883b-aa960c6dd304-dbus\") pod \"global-pull-secret-syncer-8f5jf\" (UID: \"afdb1ba0-560a-4b0f-883b-aa960c6dd304\") " pod="kube-system/global-pull-secret-syncer-8f5jf"
Jul 07 11:26:56.041578 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.041438    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4e27418c-5ace-4a75-a4b9-931fffa64650-multus-socket-dir-parent\") pod \"multus-lj4rs\" (UID: \"4e27418c-5ace-4a75-a4b9-931fffa64650\") " pod="openshift-multus/multus-lj4rs"
Jul 07 11:26:56.041578 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.041508    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4e27418c-5ace-4a75-a4b9-931fffa64650-multus-socket-dir-parent\") pod \"multus-lj4rs\" (UID: \"4e27418c-5ace-4a75-a4b9-931fffa64650\") " pod="openshift-multus/multus-lj4rs"
Jul 07 11:26:56.043097 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.042768    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b2009767-f15b-464f-8a3f-87335106fe11-run\") pod \"tuned-dljdk\" (UID: \"b2009767-f15b-464f-8a3f-87335106fe11\") " pod="openshift-cluster-node-tuning-operator/tuned-dljdk"
Jul 07 11:26:56.043097 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.042777    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4e27418c-5ace-4a75-a4b9-931fffa64650-host-var-lib-cni-multus\") pod \"multus-lj4rs\" (UID: \"4e27418c-5ace-4a75-a4b9-931fffa64650\") " pod="openshift-multus/multus-lj4rs"
Jul 07 11:26:56.043885 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.043363    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b2009767-f15b-464f-8a3f-87335106fe11-lib-modules\") pod \"tuned-dljdk\" (UID: \"b2009767-f15b-464f-8a3f-87335106fe11\") " pod="openshift-cluster-node-tuning-operator/tuned-dljdk"
Jul 07 11:26:56.043885 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.043408    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e54ef86d-1806-4e84-a235-aba912f88f58-host\") pod \"node-ca-qh2dm\" (UID: \"e54ef86d-1806-4e84-a235-aba912f88f58\") " pod="openshift-image-registry/node-ca-qh2dm"
Jul 07 11:26:56.043885 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.043435    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4e27418c-5ace-4a75-a4b9-931fffa64650-hostroot\") pod \"multus-lj4rs\" (UID: \"4e27418c-5ace-4a75-a4b9-931fffa64650\") " pod="openshift-multus/multus-lj4rs"
Jul 07 11:26:56.043885 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:56.043455    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 11:26:56.043885 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.043470    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4e27418c-5ace-4a75-a4b9-931fffa64650-etc-kubernetes\") pod \"multus-lj4rs\" (UID: \"4e27418c-5ace-4a75-a4b9-931fffa64650\") " pod="openshift-multus/multus-lj4rs"
Jul 07 11:26:56.043885 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.043515    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b2009767-f15b-464f-8a3f-87335106fe11-host\") pod \"tuned-dljdk\" (UID: \"b2009767-f15b-464f-8a3f-87335106fe11\") " pod="openshift-cluster-node-tuning-operator/tuned-dljdk"
Jul 07 11:26:56.043885 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.043578    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b2009767-f15b-464f-8a3f-87335106fe11-host\") pod \"tuned-dljdk\" (UID: \"b2009767-f15b-464f-8a3f-87335106fe11\") " pod="openshift-cluster-node-tuning-operator/tuned-dljdk"
Jul 07 11:26:56.043885 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:56.043570    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f75e11d0-c0de-4a8d-849d-807a87aa1497-metrics-certs podName:f75e11d0-c0de-4a8d-849d-807a87aa1497 nodeName:}" failed. No retries permitted until 2026-07-07 11:26:56.543523368 +0000 UTC m=+4.517163779 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f75e11d0-c0de-4a8d-849d-807a87aa1497-metrics-certs") pod "network-metrics-daemon-ppmbz" (UID: "f75e11d0-c0de-4a8d-849d-807a87aa1497") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 11:26:56.043885 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.043642    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4e27418c-5ace-4a75-a4b9-931fffa64650-host-run-netns\") pod \"multus-lj4rs\" (UID: \"4e27418c-5ace-4a75-a4b9-931fffa64650\") " pod="openshift-multus/multus-lj4rs"
Jul 07 11:26:56.043885 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.043725    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ea85da56-a626-4fc7-a0d4-31b668987496-etc-selinux\") pod \"aws-ebs-csi-driver-node-lmxhq\" (UID: \"ea85da56-a626-4fc7-a0d4-31b668987496\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lmxhq"
Jul 07 11:26:56.043885 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.043753    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4e27418c-5ace-4a75-a4b9-931fffa64650-hostroot\") pod \"multus-lj4rs\" (UID: \"4e27418c-5ace-4a75-a4b9-931fffa64650\") " pod="openshift-multus/multus-lj4rs"
Jul 07 11:26:56.043885 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.043797    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4e27418c-5ace-4a75-a4b9-931fffa64650-etc-kubernetes\") pod \"multus-lj4rs\" (UID: \"4e27418c-5ace-4a75-a4b9-931fffa64650\") " pod="openshift-multus/multus-lj4rs"
Jul 07 11:26:56.043885 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.043801    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b2009767-f15b-464f-8a3f-87335106fe11-lib-modules\") pod \"tuned-dljdk\" (UID: \"b2009767-f15b-464f-8a3f-87335106fe11\") " pod="openshift-cluster-node-tuning-operator/tuned-dljdk"
Jul 07 11:26:56.043885 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.043841    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/afdb1ba0-560a-4b0f-883b-aa960c6dd304-kubelet-config\") pod \"global-pull-secret-syncer-8f5jf\" (UID: \"afdb1ba0-560a-4b0f-883b-aa960c6dd304\") " pod="kube-system/global-pull-secret-syncer-8f5jf"
Jul 07 11:26:56.044759 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.043905    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4e27418c-5ace-4a75-a4b9-931fffa64650-host-run-netns\") pod \"multus-lj4rs\" (UID: \"4e27418c-5ace-4a75-a4b9-931fffa64650\") " pod="openshift-multus/multus-lj4rs"
Jul 07 11:26:56.044759 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.043913    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4e27418c-5ace-4a75-a4b9-931fffa64650-system-cni-dir\") pod \"multus-lj4rs\" (UID: \"4e27418c-5ace-4a75-a4b9-931fffa64650\") " pod="openshift-multus/multus-lj4rs"
Jul 07 11:26:56.044759 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.043923    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b2009767-f15b-464f-8a3f-87335106fe11-etc-systemd\") pod \"tuned-dljdk\" (UID: \"b2009767-f15b-464f-8a3f-87335106fe11\") " pod="openshift-cluster-node-tuning-operator/tuned-dljdk"
Jul 07 11:26:56.044759 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.043976    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b2009767-f15b-464f-8a3f-87335106fe11-etc-modprobe-d\") pod \"tuned-dljdk\" (UID: \"b2009767-f15b-464f-8a3f-87335106fe11\") " pod="openshift-cluster-node-tuning-operator/tuned-dljdk"
Jul 07 11:26:56.044759 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.044025    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b2009767-f15b-464f-8a3f-87335106fe11-etc-kubernetes\") pod \"tuned-dljdk\" (UID: \"b2009767-f15b-464f-8a3f-87335106fe11\") " pod="openshift-cluster-node-tuning-operator/tuned-dljdk"
Jul 07 11:26:56.044759 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.044032    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b2009767-f15b-464f-8a3f-87335106fe11-etc-sysctl-d\") pod \"tuned-dljdk\" (UID: \"b2009767-f15b-464f-8a3f-87335106fe11\") " pod="openshift-cluster-node-tuning-operator/tuned-dljdk"
Jul 07 11:26:56.044759 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.044037    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e54ef86d-1806-4e84-a235-aba912f88f58-host\") pod \"node-ca-qh2dm\" (UID: \"e54ef86d-1806-4e84-a235-aba912f88f58\") " pod="openshift-image-registry/node-ca-qh2dm"
Jul 07 11:26:56.044759 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.044043    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b2009767-f15b-464f-8a3f-87335106fe11-etc-modprobe-d\") pod \"tuned-dljdk\" (UID: \"b2009767-f15b-464f-8a3f-87335106fe11\") " pod="openshift-cluster-node-tuning-operator/tuned-dljdk"
Jul 07 11:26:56.044759 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.044092    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b2009767-f15b-464f-8a3f-87335106fe11-etc-kubernetes\") pod \"tuned-dljdk\" (UID: \"b2009767-f15b-464f-8a3f-87335106fe11\") " pod="openshift-cluster-node-tuning-operator/tuned-dljdk"
Jul 07 11:26:56.044759 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.044131    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b2009767-f15b-464f-8a3f-87335106fe11-etc-sysctl-conf\") pod \"tuned-dljdk\" (UID: \"b2009767-f15b-464f-8a3f-87335106fe11\") " pod="openshift-cluster-node-tuning-operator/tuned-dljdk"
Jul 07 11:26:56.044759 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.044233    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4e27418c-5ace-4a75-a4b9-931fffa64650-multus-conf-dir\") pod \"multus-lj4rs\" (UID: \"4e27418c-5ace-4a75-a4b9-931fffa64650\") " pod="openshift-multus/multus-lj4rs"
Jul 07 11:26:56.044759 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.044314    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4e27418c-5ace-4a75-a4b9-931fffa64650-cnibin\") pod \"multus-lj4rs\" (UID: \"4e27418c-5ace-4a75-a4b9-931fffa64650\") " pod="openshift-multus/multus-lj4rs"
Jul 07 11:26:56.044759 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.044371    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/afdb1ba0-560a-4b0f-883b-aa960c6dd304-kubelet-config\") pod \"global-pull-secret-syncer-8f5jf\" (UID: \"afdb1ba0-560a-4b0f-883b-aa960c6dd304\") " pod="kube-system/global-pull-secret-syncer-8f5jf"
Jul 07 11:26:56.044759 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.044433    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpvn2\" (UniqueName: \"kubernetes.io/projected/b2009767-f15b-464f-8a3f-87335106fe11-kube-api-access-mpvn2\") pod \"tuned-dljdk\" (UID: \"b2009767-f15b-464f-8a3f-87335106fe11\") " pod="openshift-cluster-node-tuning-operator/tuned-dljdk"
Jul 07 11:26:56.044759 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.044474    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ea85da56-a626-4fc7-a0d4-31b668987496-device-dir\") pod \"aws-ebs-csi-driver-node-lmxhq\" (UID: \"ea85da56-a626-4fc7-a0d4-31b668987496\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lmxhq"
Jul 07 11:26:56.044759 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.044512    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ea85da56-a626-4fc7-a0d4-31b668987496-registration-dir\") pod \"aws-ebs-csi-driver-node-lmxhq\" (UID: \"ea85da56-a626-4fc7-a0d4-31b668987496\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lmxhq"
Jul 07 11:26:56.044759 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.044548    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ea85da56-a626-4fc7-a0d4-31b668987496-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lmxhq\" (UID: \"ea85da56-a626-4fc7-a0d4-31b668987496\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lmxhq"
Jul 07 11:26:56.044759 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.044587    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4e27418c-5ace-4a75-a4b9-931fffa64650-host-var-lib-cni-bin\") pod \"multus-lj4rs\" (UID: \"4e27418c-5ace-4a75-a4b9-931fffa64650\") " pod="openshift-multus/multus-lj4rs"
Jul 07 11:26:56.045664 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.044638    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4e27418c-5ace-4a75-a4b9-931fffa64650-host-var-lib-cni-bin\") pod \"multus-lj4rs\" (UID: \"4e27418c-5ace-4a75-a4b9-931fffa64650\") " pod="openshift-multus/multus-lj4rs"
Jul 07 11:26:56.045664 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.044665    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4e27418c-5ace-4a75-a4b9-931fffa64650-host-run-multus-certs\") pod \"multus-lj4rs\" (UID: \"4e27418c-5ace-4a75-a4b9-931fffa64650\") " pod="openshift-multus/multus-lj4rs"
Jul 07 11:26:56.045664 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.044696    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xv7jm\" (UniqueName: \"kubernetes.io/projected/4e27418c-5ace-4a75-a4b9-931fffa64650-kube-api-access-xv7jm\") pod \"multus-lj4rs\" (UID: \"4e27418c-5ace-4a75-a4b9-931fffa64650\") " pod="openshift-multus/multus-lj4rs"
Jul 07 11:26:56.045664 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.044707    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ea85da56-a626-4fc7-a0d4-31b668987496-device-dir\") pod \"aws-ebs-csi-driver-node-lmxhq\" (UID: \"ea85da56-a626-4fc7-a0d4-31b668987496\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lmxhq"
Jul 07 11:26:56.045664 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.044732    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/af613b4a-9c2e-4b9b-8a10-0ca67f1f9159-konnectivity-ca\") pod \"konnectivity-agent-7tk6n\" (UID: \"af613b4a-9c2e-4b9b-8a10-0ca67f1f9159\") " pod="kube-system/konnectivity-agent-7tk6n"
Jul 07 11:26:56.045664 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.044766    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e54ef86d-1806-4e84-a235-aba912f88f58-serviceca\") pod \"node-ca-qh2dm\" (UID: \"e54ef86d-1806-4e84-a235-aba912f88f58\") " pod="openshift-image-registry/node-ca-qh2dm"
Jul 07 11:26:56.045664 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.044766    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ea85da56-a626-4fc7-a0d4-31b668987496-registration-dir\") pod \"aws-ebs-csi-driver-node-lmxhq\" (UID: \"ea85da56-a626-4fc7-a0d4-31b668987496\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lmxhq"
Jul 07 11:26:56.045664 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.044841    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ea85da56-a626-4fc7-a0d4-31b668987496-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lmxhq\" (UID: \"ea85da56-a626-4fc7-a0d4-31b668987496\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lmxhq"
Jul 07 11:26:56.045664 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.044878    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4e27418c-5ace-4a75-a4b9-931fffa64650-multus-cni-dir\") pod \"multus-lj4rs\" (UID: \"4e27418c-5ace-4a75-a4b9-931fffa64650\") " pod="openshift-multus/multus-lj4rs"
Jul 07 11:26:56.045664 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.044916    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8chf\" (UniqueName: \"kubernetes.io/projected/f75e11d0-c0de-4a8d-849d-807a87aa1497-kube-api-access-w8chf\") pod \"network-metrics-daemon-ppmbz\" (UID: \"f75e11d0-c0de-4a8d-849d-807a87aa1497\") " pod="openshift-multus/network-metrics-daemon-ppmbz"
Jul 07 11:26:56.045664 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.044943    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b2009767-f15b-464f-8a3f-87335106fe11-var-lib-kubelet\") pod \"tuned-dljdk\" (UID: \"b2009767-f15b-464f-8a3f-87335106fe11\") " pod="openshift-cluster-node-tuning-operator/tuned-dljdk"
Jul 07 11:26:56.045664 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.044975    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ea85da56-a626-4fc7-a0d4-31b668987496-socket-dir\") pod \"aws-ebs-csi-driver-node-lmxhq\" (UID: \"ea85da56-a626-4fc7-a0d4-31b668987496\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lmxhq"
Jul 07 11:26:56.045664 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.045025    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/afdb1ba0-560a-4b0f-883b-aa960c6dd304-original-pull-secret\") pod \"global-pull-secret-syncer-8f5jf\" (UID: \"afdb1ba0-560a-4b0f-883b-aa960c6dd304\") " pod="kube-system/global-pull-secret-syncer-8f5jf"
Jul 07 11:26:56.045664 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.045061    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/af613b4a-9c2e-4b9b-8a10-0ca67f1f9159-agent-certs\") pod \"konnectivity-agent-7tk6n\" (UID: \"af613b4a-9c2e-4b9b-8a10-0ca67f1f9159\") " pod="kube-system/konnectivity-agent-7tk6n"
Jul 07 11:26:56.045664 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.045093    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hb26c\" (UniqueName: \"kubernetes.io/projected/732e808f-49d9-44a1-b93f-1cdb95c31e0b-kube-api-access-hb26c\") pod \"network-check-target-7bk7n\" (UID: \"732e808f-49d9-44a1-b93f-1cdb95c31e0b\") " pod="openshift-network-diagnostics/network-check-target-7bk7n"
Jul 07 11:26:56.045664 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.045169    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4e27418c-5ace-4a75-a4b9-931fffa64650-cni-binary-copy\") pod \"multus-lj4rs\" (UID: \"4e27418c-5ace-4a75-a4b9-931fffa64650\") " pod="openshift-multus/multus-lj4rs"
Jul 07 11:26:56.045664 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.045181    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b2009767-f15b-464f-8a3f-87335106fe11-etc-sysconfig\") pod \"tuned-dljdk\" (UID: \"b2009767-f15b-464f-8a3f-87335106fe11\") " pod="openshift-cluster-node-tuning-operator/tuned-dljdk"
Jul 07 11:26:56.046588 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.045283    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b2009767-f15b-464f-8a3f-87335106fe11-etc-sysconfig\") pod \"tuned-dljdk\" (UID: \"b2009767-f15b-464f-8a3f-87335106fe11\") " pod="openshift-cluster-node-tuning-operator/tuned-dljdk"
Jul 07 11:26:56.046588 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.045433    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ea85da56-a626-4fc7-a0d4-31b668987496-socket-dir\") pod \"aws-ebs-csi-driver-node-lmxhq\" (UID: \"ea85da56-a626-4fc7-a0d4-31b668987496\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lmxhq"
Jul 07 11:26:56.046588 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.045525    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b2009767-f15b-464f-8a3f-87335106fe11-sys\") pod \"tuned-dljdk\" (UID: \"b2009767-f15b-464f-8a3f-87335106fe11\") " pod="openshift-cluster-node-tuning-operator/tuned-dljdk"
Jul 07 11:26:56.046588 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.045687    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e54ef86d-1806-4e84-a235-aba912f88f58-serviceca\") pod \"node-ca-qh2dm\" (UID: \"e54ef86d-1806-4e84-a235-aba912f88f58\") " pod="openshift-image-registry/node-ca-qh2dm"
Jul 07 11:26:56.046588 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:56.045722    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 11:26:56.046588 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.045913    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b2009767-f15b-464f-8a3f-87335106fe11-etc-sysctl-conf\") pod \"tuned-dljdk\" (UID: \"b2009767-f15b-464f-8a3f-87335106fe11\") " pod="openshift-cluster-node-tuning-operator/tuned-dljdk"
Jul 07 11:26:56.046588 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.046102    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4e27418c-5ace-4a75-a4b9-931fffa64650-multus-daemon-config\") pod \"multus-lj4rs\" (UID: \"4e27418c-5ace-4a75-a4b9-931fffa64650\") " pod="openshift-multus/multus-lj4rs"
Jul 07 11:26:56.046588 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.046129    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4e27418c-5ace-4a75-a4b9-931fffa64650-host-run-multus-certs\") pod \"multus-lj4rs\" (UID: \"4e27418c-5ace-4a75-a4b9-931fffa64650\") " pod="openshift-multus/multus-lj4rs"
Jul 07 11:26:56.046588 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.046244    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b2009767-f15b-464f-8a3f-87335106fe11-sys\") pod \"tuned-dljdk\" (UID: \"b2009767-f15b-464f-8a3f-87335106fe11\") " pod="openshift-cluster-node-tuning-operator/tuned-dljdk"
Jul 07 11:26:56.046588 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.046267    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b2009767-f15b-464f-8a3f-87335106fe11-tmp\") pod \"tuned-dljdk\" (UID: \"b2009767-f15b-464f-8a3f-87335106fe11\") " pod="openshift-cluster-node-tuning-operator/tuned-dljdk"
Jul 07 11:26:56.046588 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.046326    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4e27418c-5ace-4a75-a4b9-931fffa64650-multus-cni-dir\") pod \"multus-lj4rs\" (UID: \"4e27418c-5ace-4a75-a4b9-931fffa64650\") " pod="openshift-multus/multus-lj4rs"
Jul 07 11:26:56.046588 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:56.046405    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afdb1ba0-560a-4b0f-883b-aa960c6dd304-original-pull-secret podName:afdb1ba0-560a-4b0f-883b-aa960c6dd304 nodeName:}" failed. No retries permitted until 2026-07-07 11:26:56.546383693 +0000 UTC m=+4.520024095 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/afdb1ba0-560a-4b0f-883b-aa960c6dd304-original-pull-secret") pod "global-pull-secret-syncer-8f5jf" (UID: "afdb1ba0-560a-4b0f-883b-aa960c6dd304") : object "kube-system"/"original-pull-secret" not registered
Jul 07 11:26:56.046588 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.046462    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b2009767-f15b-464f-8a3f-87335106fe11-var-lib-kubelet\") pod \"tuned-dljdk\" (UID: \"b2009767-f15b-464f-8a3f-87335106fe11\") " pod="openshift-cluster-node-tuning-operator/tuned-dljdk"
Jul 07 11:26:56.047327 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.046955    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/af613b4a-9c2e-4b9b-8a10-0ca67f1f9159-konnectivity-ca\") pod \"konnectivity-agent-7tk6n\" (UID: \"af613b4a-9c2e-4b9b-8a10-0ca67f1f9159\") " pod="kube-system/konnectivity-agent-7tk6n"
Jul 07 11:26:56.048121 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.048034    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b2009767-f15b-464f-8a3f-87335106fe11-etc-tuned\") pod \"tuned-dljdk\" (UID: \"b2009767-f15b-464f-8a3f-87335106fe11\") " pod="openshift-cluster-node-tuning-operator/tuned-dljdk"
Jul 07 11:26:56.048463 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.048421    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/af613b4a-9c2e-4b9b-8a10-0ca67f1f9159-agent-certs\") pod \"konnectivity-agent-7tk6n\" (UID: \"af613b4a-9c2e-4b9b-8a10-0ca67f1f9159\") " pod="kube-system/konnectivity-agent-7tk6n"
Jul 07 11:26:56.051920 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.051893    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttqsw\" (UniqueName: \"kubernetes.io/projected/ea85da56-a626-4fc7-a0d4-31b668987496-kube-api-access-ttqsw\") pod \"aws-ebs-csi-driver-node-lmxhq\" (UID: \"ea85da56-a626-4fc7-a0d4-31b668987496\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lmxhq"
Jul 07 11:26:56.052613 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.052551    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-khf5x\" (UniqueName: \"kubernetes.io/projected/e54ef86d-1806-4e84-a235-aba912f88f58-kube-api-access-khf5x\") pod \"node-ca-qh2dm\" (UID: \"e54ef86d-1806-4e84-a235-aba912f88f58\") " pod="openshift-image-registry/node-ca-qh2dm"
Jul 07 11:26:56.052858 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:56.052761    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 11:26:56.052858 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:56.052783    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 11:26:56.052858 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:56.052799    2570 projected.go:196] Error preparing data for projected volume kube-api-access-hb26c for pod openshift-network-diagnostics/network-check-target-7bk7n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 11:26:56.053080 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:56.052869    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/732e808f-49d9-44a1-b93f-1cdb95c31e0b-kube-api-access-hb26c podName:732e808f-49d9-44a1-b93f-1cdb95c31e0b nodeName:}" failed. No retries permitted until 2026-07-07 11:26:56.552852455 +0000 UTC m=+4.526492866 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-hb26c" (UniqueName: "kubernetes.io/projected/732e808f-49d9-44a1-b93f-1cdb95c31e0b-kube-api-access-hb26c") pod "network-check-target-7bk7n" (UID: "732e808f-49d9-44a1-b93f-1cdb95c31e0b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 11:26:56.055167 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.055116    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpvn2\" (UniqueName: \"kubernetes.io/projected/b2009767-f15b-464f-8a3f-87335106fe11-kube-api-access-mpvn2\") pod \"tuned-dljdk\" (UID: \"b2009767-f15b-464f-8a3f-87335106fe11\") " pod="openshift-cluster-node-tuning-operator/tuned-dljdk"
Jul 07 11:26:56.056080 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.056020    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xv7jm\" (UniqueName: \"kubernetes.io/projected/4e27418c-5ace-4a75-a4b9-931fffa64650-kube-api-access-xv7jm\") pod \"multus-lj4rs\" (UID: \"4e27418c-5ace-4a75-a4b9-931fffa64650\") " pod="openshift-multus/multus-lj4rs"
Jul 07 11:26:56.056080 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.056063    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8chf\" (UniqueName: \"kubernetes.io/projected/f75e11d0-c0de-4a8d-849d-807a87aa1497-kube-api-access-w8chf\") pod \"network-metrics-daemon-ppmbz\" (UID: \"f75e11d0-c0de-4a8d-849d-807a87aa1497\") " pod="openshift-multus/network-metrics-daemon-ppmbz"
Jul 07 11:26:56.133535 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.133427    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8f25d"
Jul 07 11:26:56.141445 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.141338    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rkn2t"
Jul 07 11:26:56.154590 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.154474    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vdrl5"
Jul 07 11:26:56.161447 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.161408    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:26:56.170644 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.170618    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7tk6n"
Jul 07 11:26:56.179058 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.179033    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qh2dm"
Jul 07 11:26:56.188966 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.188937    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lmxhq"
Jul 07 11:26:56.196925 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.196898    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lj4rs"
Jul 07 11:26:56.202901 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.202860    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dljdk"
Jul 07 11:26:56.549590 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.549546    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/afdb1ba0-560a-4b0f-883b-aa960c6dd304-original-pull-secret\") pod \"global-pull-secret-syncer-8f5jf\" (UID: \"afdb1ba0-560a-4b0f-883b-aa960c6dd304\") " pod="kube-system/global-pull-secret-syncer-8f5jf"
Jul 07 11:26:56.549821 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.549622    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f75e11d0-c0de-4a8d-849d-807a87aa1497-metrics-certs\") pod \"network-metrics-daemon-ppmbz\" (UID: \"f75e11d0-c0de-4a8d-849d-807a87aa1497\") " pod="openshift-multus/network-metrics-daemon-ppmbz"
Jul 07 11:26:56.549821 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:56.549727    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 11:26:56.549821 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:56.549808    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f75e11d0-c0de-4a8d-849d-807a87aa1497-metrics-certs podName:f75e11d0-c0de-4a8d-849d-807a87aa1497 nodeName:}" failed. No retries permitted until 2026-07-07 11:26:57.549786864 +0000 UTC m=+5.523427282 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f75e11d0-c0de-4a8d-849d-807a87aa1497-metrics-certs") pod "network-metrics-daemon-ppmbz" (UID: "f75e11d0-c0de-4a8d-849d-807a87aa1497") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 11:26:56.550012 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:56.549727    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 11:26:56.550012 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:56.549900    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afdb1ba0-560a-4b0f-883b-aa960c6dd304-original-pull-secret podName:afdb1ba0-560a-4b0f-883b-aa960c6dd304 nodeName:}" failed. No retries permitted until 2026-07-07 11:26:57.54988312 +0000 UTC m=+5.523523534 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/afdb1ba0-560a-4b0f-883b-aa960c6dd304-original-pull-secret") pod "global-pull-secret-syncer-8f5jf" (UID: "afdb1ba0-560a-4b0f-883b-aa960c6dd304") : object "kube-system"/"original-pull-secret" not registered
Jul 07 11:26:56.650392 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.650341    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hb26c\" (UniqueName: \"kubernetes.io/projected/732e808f-49d9-44a1-b93f-1cdb95c31e0b-kube-api-access-hb26c\") pod \"network-check-target-7bk7n\" (UID: \"732e808f-49d9-44a1-b93f-1cdb95c31e0b\") " pod="openshift-network-diagnostics/network-check-target-7bk7n"
Jul 07 11:26:56.650585 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:56.650560    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 11:26:56.650645 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:56.650595    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 11:26:56.650645 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:56.650607    2570 projected.go:196] Error preparing data for projected volume kube-api-access-hb26c for pod openshift-network-diagnostics/network-check-target-7bk7n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 11:26:56.650735 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:56.650673    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/732e808f-49d9-44a1-b93f-1cdb95c31e0b-kube-api-access-hb26c podName:732e808f-49d9-44a1-b93f-1cdb95c31e0b nodeName:}" failed. No retries permitted until 2026-07-07 11:26:57.650654283 +0000 UTC m=+5.624294692 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-hb26c" (UniqueName: "kubernetes.io/projected/732e808f-49d9-44a1-b93f-1cdb95c31e0b-kube-api-access-hb26c") pod "network-check-target-7bk7n" (UID: "732e808f-49d9-44a1-b93f-1cdb95c31e0b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 11:26:56.878783 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.878678    2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 11:21:56 +0000 UTC" deadline="2027-12-17 15:12:47.247782342 +0000 UTC"
Jul 07 11:26:56.878783 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:56.878729    2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12675h45m50.369058617s"
Jul 07 11:26:57.155941 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:57.155722    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9392160_8536_4d5e_916d_e2a0164db043.slice/crio-10b93fa3c35160d6ee09869ea02026fd48eb61c15cd5fb04ae5bb5791d81bd21 WatchSource:0}: Error finding container 10b93fa3c35160d6ee09869ea02026fd48eb61c15cd5fb04ae5bb5791d81bd21: Status 404 returned error can't find the container with id 10b93fa3c35160d6ee09869ea02026fd48eb61c15cd5fb04ae5bb5791d81bd21
Jul 07 11:26:57.157690 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:57.157661    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea85da56_a626_4fc7_a0d4_31b668987496.slice/crio-c78302d4f9ab411f93d9e4a3e0ea22bc3a3ce713cce228b54e5888c2978e4239 WatchSource:0}: Error finding container c78302d4f9ab411f93d9e4a3e0ea22bc3a3ce713cce228b54e5888c2978e4239: Status 404 returned error can't find the container with id c78302d4f9ab411f93d9e4a3e0ea22bc3a3ce713cce228b54e5888c2978e4239
Jul 07 11:26:57.161601 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:57.161574    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20c2dc90_b7d3_4add_af1f_31f1ba991c54.slice/crio-1728dd79ffd96b7f7ecfd4b757b6e62880fddbf3cce5bf0cfb762c9a3896bf18 WatchSource:0}: Error finding container 1728dd79ffd96b7f7ecfd4b757b6e62880fddbf3cce5bf0cfb762c9a3896bf18: Status 404 returned error can't find the container with id 1728dd79ffd96b7f7ecfd4b757b6e62880fddbf3cce5bf0cfb762c9a3896bf18
Jul 07 11:26:57.163133 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:57.163109    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod650c2a15_194e_47f4_99c8_9f94802d1e9b.slice/crio-fbb2259aa80ed39589184f160c0c1ef97168912ff2942ba7e97e84796ce42d41 WatchSource:0}: Error finding container fbb2259aa80ed39589184f160c0c1ef97168912ff2942ba7e97e84796ce42d41: Status 404 returned error can't find the container with id fbb2259aa80ed39589184f160c0c1ef97168912ff2942ba7e97e84796ce42d41
Jul 07 11:26:57.164146 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:57.164110    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2009767_f15b_464f_8a3f_87335106fe11.slice/crio-89bb78bf249a7db96cfec84e3db5a216fe85526e308227a4cdb9dda309b5d698 WatchSource:0}: Error finding container 89bb78bf249a7db96cfec84e3db5a216fe85526e308227a4cdb9dda309b5d698: Status 404 returned error can't find the container with id 89bb78bf249a7db96cfec84e3db5a216fe85526e308227a4cdb9dda309b5d698
Jul 07 11:26:57.166155 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:57.166096    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e27418c_5ace_4a75_a4b9_931fffa64650.slice/crio-ced9cef0a99969b66b04b41eaa3406b5b88af0c4030186dd3a1fdedb282caaea WatchSource:0}: Error finding container ced9cef0a99969b66b04b41eaa3406b5b88af0c4030186dd3a1fdedb282caaea: Status 404 returned error can't find the container with id ced9cef0a99969b66b04b41eaa3406b5b88af0c4030186dd3a1fdedb282caaea
Jul 07 11:26:57.168010 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:57.167207    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf613b4a_9c2e_4b9b_8a10_0ca67f1f9159.slice/crio-056eeb5047a807cc49f0ffb7337da3691be19aa5af2813c02f9d35e9c00ece48 WatchSource:0}: Error finding container 056eeb5047a807cc49f0ffb7337da3691be19aa5af2813c02f9d35e9c00ece48: Status 404 returned error can't find the container with id 056eeb5047a807cc49f0ffb7337da3691be19aa5af2813c02f9d35e9c00ece48
Jul 07 11:26:57.168591 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:57.168569    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode54ef86d_1806_4e84_a235_aba912f88f58.slice/crio-7f6a96d06e2cbb67652d335299ea7969f25e8286e8732e02648c2fc18aa04847 WatchSource:0}: Error finding container 7f6a96d06e2cbb67652d335299ea7969f25e8286e8732e02648c2fc18aa04847: Status 404 returned error can't find the container with id 7f6a96d06e2cbb67652d335299ea7969f25e8286e8732e02648c2fc18aa04847
Jul 07 11:26:57.169595 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:26:57.169529    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6400d4cc_3b3b_4fd0_aa78_a8a3e007a36b.slice/crio-c6ba6a671ca161709cb1e5b5bc9e5e882334f0d5d1cc9b5196355131c9304602 WatchSource:0}: Error finding container c6ba6a671ca161709cb1e5b5bc9e5e882334f0d5d1cc9b5196355131c9304602: Status 404 returned error can't find the container with id c6ba6a671ca161709cb1e5b5bc9e5e882334f0d5d1cc9b5196355131c9304602
Jul 07 11:26:57.558373 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:57.558336    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f75e11d0-c0de-4a8d-849d-807a87aa1497-metrics-certs\") pod \"network-metrics-daemon-ppmbz\" (UID: \"f75e11d0-c0de-4a8d-849d-807a87aa1497\") " pod="openshift-multus/network-metrics-daemon-ppmbz"
Jul 07 11:26:57.558578 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:57.558406    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/afdb1ba0-560a-4b0f-883b-aa960c6dd304-original-pull-secret\") pod \"global-pull-secret-syncer-8f5jf\" (UID: \"afdb1ba0-560a-4b0f-883b-aa960c6dd304\") " pod="kube-system/global-pull-secret-syncer-8f5jf"
Jul 07 11:26:57.558578 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:57.558524    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 11:26:57.558694 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:57.558585    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afdb1ba0-560a-4b0f-883b-aa960c6dd304-original-pull-secret podName:afdb1ba0-560a-4b0f-883b-aa960c6dd304 nodeName:}" failed. No retries permitted until 2026-07-07 11:26:59.558566508 +0000 UTC m=+7.532206911 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/afdb1ba0-560a-4b0f-883b-aa960c6dd304-original-pull-secret") pod "global-pull-secret-syncer-8f5jf" (UID: "afdb1ba0-560a-4b0f-883b-aa960c6dd304") : object "kube-system"/"original-pull-secret" not registered
Jul 07 11:26:57.558694 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:57.558520    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 11:26:57.558694 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:57.558663    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f75e11d0-c0de-4a8d-849d-807a87aa1497-metrics-certs podName:f75e11d0-c0de-4a8d-849d-807a87aa1497 nodeName:}" failed. No retries permitted until 2026-07-07 11:26:59.558623796 +0000 UTC m=+7.532264202 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f75e11d0-c0de-4a8d-849d-807a87aa1497-metrics-certs") pod "network-metrics-daemon-ppmbz" (UID: "f75e11d0-c0de-4a8d-849d-807a87aa1497") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 11:26:57.658935 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:57.658850    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hb26c\" (UniqueName: \"kubernetes.io/projected/732e808f-49d9-44a1-b93f-1cdb95c31e0b-kube-api-access-hb26c\") pod \"network-check-target-7bk7n\" (UID: \"732e808f-49d9-44a1-b93f-1cdb95c31e0b\") " pod="openshift-network-diagnostics/network-check-target-7bk7n"
Jul 07 11:26:57.659114 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:57.659076    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 11:26:57.659114 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:57.659109    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 11:26:57.659233 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:57.659125    2570 projected.go:196] Error preparing data for projected volume kube-api-access-hb26c for pod openshift-network-diagnostics/network-check-target-7bk7n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 11:26:57.659233 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:57.659198    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/732e808f-49d9-44a1-b93f-1cdb95c31e0b-kube-api-access-hb26c podName:732e808f-49d9-44a1-b93f-1cdb95c31e0b nodeName:}" failed. No retries permitted until 2026-07-07 11:26:59.659177863 +0000 UTC m=+7.632818274 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-hb26c" (UniqueName: "kubernetes.io/projected/732e808f-49d9-44a1-b93f-1cdb95c31e0b-kube-api-access-hb26c") pod "network-check-target-7bk7n" (UID: "732e808f-49d9-44a1-b93f-1cdb95c31e0b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 11:26:57.880245 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:57.880192    2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 11:21:56 +0000 UTC" deadline="2028-02-19 17:50:35.632253302 +0000 UTC"
Jul 07 11:26:57.880793 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:57.880259    2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14214h23m37.75200066s"
Jul 07 11:26:57.933438 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:57.933405    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8f5jf"
Jul 07 11:26:57.933634 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:57.933547    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8f5jf" podUID="afdb1ba0-560a-4b0f-883b-aa960c6dd304"
Jul 07 11:26:57.934035 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:57.934014    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ppmbz"
Jul 07 11:26:57.934156 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:57.934132    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ppmbz" podUID="f75e11d0-c0de-4a8d-849d-807a87aa1497"
Jul 07 11:26:57.936321 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:57.936158    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7bk7n"
Jul 07 11:26:57.936321 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:57.936252    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7bk7n" podUID="732e808f-49d9-44a1-b93f-1cdb95c31e0b"
Jul 07 11:26:57.954647 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:57.953668    2570 generic.go:356] "Generic (PLEG): container finished" podID="c9aa93f97f9dd2447825488095d15a9f" containerID="ddd9fa3962015968cdf599b2b5e40f2312127a6e412f2fc487c874cd36736081" exitCode=0
Jul 07 11:26:57.954647 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:57.953758    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-78.ec2.internal" event={"ID":"c9aa93f97f9dd2447825488095d15a9f","Type":"ContainerDied","Data":"ddd9fa3962015968cdf599b2b5e40f2312127a6e412f2fc487c874cd36736081"}
Jul 07 11:26:57.961434 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:57.961331    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw" event={"ID":"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b","Type":"ContainerStarted","Data":"c6ba6a671ca161709cb1e5b5bc9e5e882334f0d5d1cc9b5196355131c9304602"}
Jul 07 11:26:57.967736 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:57.967702    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7tk6n" event={"ID":"af613b4a-9c2e-4b9b-8a10-0ca67f1f9159","Type":"ContainerStarted","Data":"056eeb5047a807cc49f0ffb7337da3691be19aa5af2813c02f9d35e9c00ece48"}
Jul 07 11:26:57.974053 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:57.974014    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dljdk" event={"ID":"b2009767-f15b-464f-8a3f-87335106fe11","Type":"ContainerStarted","Data":"89bb78bf249a7db96cfec84e3db5a216fe85526e308227a4cdb9dda309b5d698"}
Jul 07 11:26:57.987902 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:57.987789    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lmxhq" event={"ID":"ea85da56-a626-4fc7-a0d4-31b668987496","Type":"ContainerStarted","Data":"c78302d4f9ab411f93d9e4a3e0ea22bc3a3ce713cce228b54e5888c2978e4239"}
Jul 07 11:26:57.991032 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:57.990940    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qh2dm" event={"ID":"e54ef86d-1806-4e84-a235-aba912f88f58","Type":"ContainerStarted","Data":"7f6a96d06e2cbb67652d335299ea7969f25e8286e8732e02648c2fc18aa04847"}
Jul 07 11:26:57.993432 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:57.993369    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lj4rs" event={"ID":"4e27418c-5ace-4a75-a4b9-931fffa64650","Type":"ContainerStarted","Data":"ced9cef0a99969b66b04b41eaa3406b5b88af0c4030186dd3a1fdedb282caaea"}
Jul 07 11:26:57.995856 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:57.995778    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rkn2t" event={"ID":"650c2a15-194e-47f4-99c8-9f94802d1e9b","Type":"ContainerStarted","Data":"fbb2259aa80ed39589184f160c0c1ef97168912ff2942ba7e97e84796ce42d41"}
Jul 07 11:26:57.998152 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:57.998074    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8f25d" event={"ID":"20c2dc90-b7d3-4add-af1f-31f1ba991c54","Type":"ContainerStarted","Data":"1728dd79ffd96b7f7ecfd4b757b6e62880fddbf3cce5bf0cfb762c9a3896bf18"}
Jul 07 11:26:58.019020 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:58.018920    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vdrl5" event={"ID":"b9392160-8536-4d5e-916d-e2a0164db043","Type":"ContainerStarted","Data":"10b93fa3c35160d6ee09869ea02026fd48eb61c15cd5fb04ae5bb5791d81bd21"}
Jul 07 11:26:58.027161 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:58.026452    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-78.ec2.internal" event={"ID":"4f2009fc30615ef1da5c98996e493d3f","Type":"ContainerStarted","Data":"29e2f6333dfc4e6ac24f9f28096b909ad90fd79185c589f34e2dd48ec0fbc06b"}
Jul 07 11:26:59.041441 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:59.041364    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-78.ec2.internal" event={"ID":"c9aa93f97f9dd2447825488095d15a9f","Type":"ContainerStarted","Data":"8b3133c2a5eb6e414cfff8755bf381a4eee27e10c3c75bc698fa5a5e7685cd7f"}
Jul 07 11:26:59.066739 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:59.065982    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-78.ec2.internal" podStartSLOduration=4.06595564 podStartE2EDuration="4.06595564s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:26:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:26:58.048146927 +0000 UTC m=+6.021787342" watchObservedRunningTime="2026-07-07 11:26:59.06595564 +0000 UTC m=+7.039596054"
Jul 07 11:26:59.575329 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:59.575266    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f75e11d0-c0de-4a8d-849d-807a87aa1497-metrics-certs\") pod \"network-metrics-daemon-ppmbz\" (UID: \"f75e11d0-c0de-4a8d-849d-807a87aa1497\") " pod="openshift-multus/network-metrics-daemon-ppmbz"
Jul 07 11:26:59.575526 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:59.575361    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/afdb1ba0-560a-4b0f-883b-aa960c6dd304-original-pull-secret\") pod \"global-pull-secret-syncer-8f5jf\" (UID: \"afdb1ba0-560a-4b0f-883b-aa960c6dd304\") " pod="kube-system/global-pull-secret-syncer-8f5jf"
Jul 07 11:26:59.575526 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:59.575512    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 11:26:59.575665 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:59.575580    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afdb1ba0-560a-4b0f-883b-aa960c6dd304-original-pull-secret podName:afdb1ba0-560a-4b0f-883b-aa960c6dd304 nodeName:}" failed. No retries permitted until 2026-07-07 11:27:03.57555854 +0000 UTC m=+11.549198934 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/afdb1ba0-560a-4b0f-883b-aa960c6dd304-original-pull-secret") pod "global-pull-secret-syncer-8f5jf" (UID: "afdb1ba0-560a-4b0f-883b-aa960c6dd304") : object "kube-system"/"original-pull-secret" not registered
Jul 07 11:26:59.576250 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:59.575995    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 11:26:59.576250 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:59.576048    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f75e11d0-c0de-4a8d-849d-807a87aa1497-metrics-certs podName:f75e11d0-c0de-4a8d-849d-807a87aa1497 nodeName:}" failed. No retries permitted until 2026-07-07 11:27:03.57603192 +0000 UTC m=+11.549672312 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f75e11d0-c0de-4a8d-849d-807a87aa1497-metrics-certs") pod "network-metrics-daemon-ppmbz" (UID: "f75e11d0-c0de-4a8d-849d-807a87aa1497") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 11:26:59.676124 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:59.676077    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hb26c\" (UniqueName: \"kubernetes.io/projected/732e808f-49d9-44a1-b93f-1cdb95c31e0b-kube-api-access-hb26c\") pod \"network-check-target-7bk7n\" (UID: \"732e808f-49d9-44a1-b93f-1cdb95c31e0b\") " pod="openshift-network-diagnostics/network-check-target-7bk7n"
Jul 07 11:26:59.676381 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:59.676280    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 11:26:59.676381 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:59.676305    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 11:26:59.676381 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:59.676319    2570 projected.go:196] Error preparing data for projected volume kube-api-access-hb26c for pod openshift-network-diagnostics/network-check-target-7bk7n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 11:26:59.676578 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:59.676386    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/732e808f-49d9-44a1-b93f-1cdb95c31e0b-kube-api-access-hb26c podName:732e808f-49d9-44a1-b93f-1cdb95c31e0b nodeName:}" failed. No retries permitted until 2026-07-07 11:27:03.676364947 +0000 UTC m=+11.650005354 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-hb26c" (UniqueName: "kubernetes.io/projected/732e808f-49d9-44a1-b93f-1cdb95c31e0b-kube-api-access-hb26c") pod "network-check-target-7bk7n" (UID: "732e808f-49d9-44a1-b93f-1cdb95c31e0b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 11:26:59.934565 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:59.934531    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8f5jf"
Jul 07 11:26:59.934766 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:59.934691    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8f5jf" podUID="afdb1ba0-560a-4b0f-883b-aa960c6dd304"
Jul 07 11:26:59.935121 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:59.935101    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ppmbz"
Jul 07 11:26:59.935247 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:59.935224    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ppmbz" podUID="f75e11d0-c0de-4a8d-849d-807a87aa1497"
Jul 07 11:26:59.935331 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:26:59.935280    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7bk7n"
Jul 07 11:26:59.935386 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:26:59.935370    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7bk7n" podUID="732e808f-49d9-44a1-b93f-1cdb95c31e0b"
Jul 07 11:27:01.932588 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:01.932551    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ppmbz"
Jul 07 11:27:01.932588 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:01.932588    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8f5jf"
Jul 07 11:27:01.933154 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:01.932551    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7bk7n"
Jul 07 11:27:01.933154 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:01.932701    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ppmbz" podUID="f75e11d0-c0de-4a8d-849d-807a87aa1497"
Jul 07 11:27:01.933154 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:01.932781    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7bk7n" podUID="732e808f-49d9-44a1-b93f-1cdb95c31e0b"
Jul 07 11:27:01.933154 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:01.932855    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8f5jf" podUID="afdb1ba0-560a-4b0f-883b-aa960c6dd304"
Jul 07 11:27:03.610929 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:03.610887    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f75e11d0-c0de-4a8d-849d-807a87aa1497-metrics-certs\") pod \"network-metrics-daemon-ppmbz\" (UID: \"f75e11d0-c0de-4a8d-849d-807a87aa1497\") " pod="openshift-multus/network-metrics-daemon-ppmbz"
Jul 07 11:27:03.611389 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:03.610964    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/afdb1ba0-560a-4b0f-883b-aa960c6dd304-original-pull-secret\") pod \"global-pull-secret-syncer-8f5jf\" (UID: \"afdb1ba0-560a-4b0f-883b-aa960c6dd304\") " pod="kube-system/global-pull-secret-syncer-8f5jf"
Jul 07 11:27:03.611389 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:03.611071    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 11:27:03.611389 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:03.611071    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 11:27:03.611389 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:03.611154    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f75e11d0-c0de-4a8d-849d-807a87aa1497-metrics-certs podName:f75e11d0-c0de-4a8d-849d-807a87aa1497 nodeName:}" failed. No retries permitted until 2026-07-07 11:27:11.611131787 +0000 UTC m=+19.584772199 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f75e11d0-c0de-4a8d-849d-807a87aa1497-metrics-certs") pod "network-metrics-daemon-ppmbz" (UID: "f75e11d0-c0de-4a8d-849d-807a87aa1497") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 11:27:03.611389 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:03.611188    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afdb1ba0-560a-4b0f-883b-aa960c6dd304-original-pull-secret podName:afdb1ba0-560a-4b0f-883b-aa960c6dd304 nodeName:}" failed. No retries permitted until 2026-07-07 11:27:11.611175408 +0000 UTC m=+19.584815803 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/afdb1ba0-560a-4b0f-883b-aa960c6dd304-original-pull-secret") pod "global-pull-secret-syncer-8f5jf" (UID: "afdb1ba0-560a-4b0f-883b-aa960c6dd304") : object "kube-system"/"original-pull-secret" not registered
Jul 07 11:27:03.711650 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:03.711610    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hb26c\" (UniqueName: \"kubernetes.io/projected/732e808f-49d9-44a1-b93f-1cdb95c31e0b-kube-api-access-hb26c\") pod \"network-check-target-7bk7n\" (UID: \"732e808f-49d9-44a1-b93f-1cdb95c31e0b\") " pod="openshift-network-diagnostics/network-check-target-7bk7n"
Jul 07 11:27:03.711830 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:03.711808    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 11:27:03.711907 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:03.711838    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 11:27:03.711907 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:03.711853    2570 projected.go:196] Error preparing data for projected volume kube-api-access-hb26c for pod openshift-network-diagnostics/network-check-target-7bk7n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 11:27:03.712010 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:03.711923    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/732e808f-49d9-44a1-b93f-1cdb95c31e0b-kube-api-access-hb26c podName:732e808f-49d9-44a1-b93f-1cdb95c31e0b nodeName:}" failed. No retries permitted until 2026-07-07 11:27:11.711903988 +0000 UTC m=+19.685544378 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-hb26c" (UniqueName: "kubernetes.io/projected/732e808f-49d9-44a1-b93f-1cdb95c31e0b-kube-api-access-hb26c") pod "network-check-target-7bk7n" (UID: "732e808f-49d9-44a1-b93f-1cdb95c31e0b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 11:27:03.932284 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:03.932247    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8f5jf"
Jul 07 11:27:03.932458 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:03.932385    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ppmbz"
Jul 07 11:27:03.932458 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:03.932389    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8f5jf" podUID="afdb1ba0-560a-4b0f-883b-aa960c6dd304"
Jul 07 11:27:03.932579 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:03.932485    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ppmbz" podUID="f75e11d0-c0de-4a8d-849d-807a87aa1497"
Jul 07 11:27:03.933511 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:03.933489    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7bk7n"
Jul 07 11:27:03.933617 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:03.933580    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7bk7n" podUID="732e808f-49d9-44a1-b93f-1cdb95c31e0b"
Jul 07 11:27:05.932836 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:05.932329    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ppmbz"
Jul 07 11:27:05.932836 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:05.932328    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8f5jf"
Jul 07 11:27:05.932836 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:05.932351    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7bk7n"
Jul 07 11:27:05.932836 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:05.932494    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ppmbz" podUID="f75e11d0-c0de-4a8d-849d-807a87aa1497"
Jul 07 11:27:05.932836 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:05.932678    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7bk7n" podUID="732e808f-49d9-44a1-b93f-1cdb95c31e0b"
Jul 07 11:27:05.932836 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:05.932771    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8f5jf" podUID="afdb1ba0-560a-4b0f-883b-aa960c6dd304"
Jul 07 11:27:07.931935 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:07.931902    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8f5jf"
Jul 07 11:27:07.932328 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:07.931913    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7bk7n"
Jul 07 11:27:07.932328 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:07.932020    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8f5jf" podUID="afdb1ba0-560a-4b0f-883b-aa960c6dd304"
Jul 07 11:27:07.932328 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:07.931913    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ppmbz"
Jul 07 11:27:07.932328 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:07.932081    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7bk7n" podUID="732e808f-49d9-44a1-b93f-1cdb95c31e0b"
Jul 07 11:27:07.932328 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:07.932234    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ppmbz" podUID="f75e11d0-c0de-4a8d-849d-807a87aa1497"
Jul 07 11:27:09.931936 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:09.931896    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7bk7n"
Jul 07 11:27:09.932379 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:09.931894    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ppmbz"
Jul 07 11:27:09.932379 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:09.932021    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7bk7n" podUID="732e808f-49d9-44a1-b93f-1cdb95c31e0b"
Jul 07 11:27:09.932379 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:09.931894    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8f5jf"
Jul 07 11:27:09.932379 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:09.932128    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ppmbz" podUID="f75e11d0-c0de-4a8d-849d-807a87aa1497"
Jul 07 11:27:09.932379 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:09.932172    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8f5jf" podUID="afdb1ba0-560a-4b0f-883b-aa960c6dd304"
Jul 07 11:27:11.676619 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:11.676538    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/afdb1ba0-560a-4b0f-883b-aa960c6dd304-original-pull-secret\") pod \"global-pull-secret-syncer-8f5jf\" (UID: \"afdb1ba0-560a-4b0f-883b-aa960c6dd304\") " pod="kube-system/global-pull-secret-syncer-8f5jf"
Jul 07 11:27:11.676619 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:11.676622    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f75e11d0-c0de-4a8d-849d-807a87aa1497-metrics-certs\") pod \"network-metrics-daemon-ppmbz\" (UID: \"f75e11d0-c0de-4a8d-849d-807a87aa1497\") " pod="openshift-multus/network-metrics-daemon-ppmbz"
Jul 07 11:27:11.677272 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:11.676686    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 11:27:11.677272 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:11.676730    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 11:27:11.677272 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:11.676758    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afdb1ba0-560a-4b0f-883b-aa960c6dd304-original-pull-secret podName:afdb1ba0-560a-4b0f-883b-aa960c6dd304 nodeName:}" failed. No retries permitted until 2026-07-07 11:27:27.676736043 +0000 UTC m=+35.650376434 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/afdb1ba0-560a-4b0f-883b-aa960c6dd304-original-pull-secret") pod "global-pull-secret-syncer-8f5jf" (UID: "afdb1ba0-560a-4b0f-883b-aa960c6dd304") : object "kube-system"/"original-pull-secret" not registered
Jul 07 11:27:11.677272 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:11.676783    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f75e11d0-c0de-4a8d-849d-807a87aa1497-metrics-certs podName:f75e11d0-c0de-4a8d-849d-807a87aa1497 nodeName:}" failed. No retries permitted until 2026-07-07 11:27:27.676765926 +0000 UTC m=+35.650406325 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f75e11d0-c0de-4a8d-849d-807a87aa1497-metrics-certs") pod "network-metrics-daemon-ppmbz" (UID: "f75e11d0-c0de-4a8d-849d-807a87aa1497") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 11:27:11.777745 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:11.777713    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hb26c\" (UniqueName: \"kubernetes.io/projected/732e808f-49d9-44a1-b93f-1cdb95c31e0b-kube-api-access-hb26c\") pod \"network-check-target-7bk7n\" (UID: \"732e808f-49d9-44a1-b93f-1cdb95c31e0b\") " pod="openshift-network-diagnostics/network-check-target-7bk7n"
Jul 07 11:27:11.777927 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:11.777840    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 11:27:11.777927 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:11.777861    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 11:27:11.777927 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:11.777870    2570 projected.go:196] Error preparing data for projected volume kube-api-access-hb26c for pod openshift-network-diagnostics/network-check-target-7bk7n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 11:27:11.777927 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:11.777915    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/732e808f-49d9-44a1-b93f-1cdb95c31e0b-kube-api-access-hb26c podName:732e808f-49d9-44a1-b93f-1cdb95c31e0b nodeName:}" failed. No retries permitted until 2026-07-07 11:27:27.777900284 +0000 UTC m=+35.751540672 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-hb26c" (UniqueName: "kubernetes.io/projected/732e808f-49d9-44a1-b93f-1cdb95c31e0b-kube-api-access-hb26c") pod "network-check-target-7bk7n" (UID: "732e808f-49d9-44a1-b93f-1cdb95c31e0b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 11:27:11.932084 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:11.932035    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8f5jf"
Jul 07 11:27:11.932294 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:11.932047    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7bk7n"
Jul 07 11:27:11.932294 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:11.932177    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8f5jf" podUID="afdb1ba0-560a-4b0f-883b-aa960c6dd304"
Jul 07 11:27:11.932294 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:11.932048    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ppmbz"
Jul 07 11:27:11.932294 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:11.932256    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7bk7n" podUID="732e808f-49d9-44a1-b93f-1cdb95c31e0b"
Jul 07 11:27:11.932536 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:11.932311    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ppmbz" podUID="f75e11d0-c0de-4a8d-849d-807a87aa1497"
Jul 07 11:27:13.933393 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:13.933356    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ppmbz"
Jul 07 11:27:13.933860 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:13.933536    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7bk7n"
Jul 07 11:27:13.933860 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:13.933556    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8f5jf"
Jul 07 11:27:13.933860 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:13.933542    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ppmbz" podUID="f75e11d0-c0de-4a8d-849d-807a87aa1497"
Jul 07 11:27:13.933860 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:13.933664    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7bk7n" podUID="732e808f-49d9-44a1-b93f-1cdb95c31e0b"
Jul 07 11:27:13.933860 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:13.933758    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8f5jf" podUID="afdb1ba0-560a-4b0f-883b-aa960c6dd304"
Jul 07 11:27:15.932227 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:15.932185    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8f5jf"
Jul 07 11:27:15.932828 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:15.932185    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7bk7n"
Jul 07 11:27:15.932828 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:15.932325    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8f5jf" podUID="afdb1ba0-560a-4b0f-883b-aa960c6dd304"
Jul 07 11:27:15.932828 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:15.932373    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7bk7n" podUID="732e808f-49d9-44a1-b93f-1cdb95c31e0b"
Jul 07 11:27:15.932828 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:15.932187    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ppmbz"
Jul 07 11:27:15.932828 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:15.932513    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ppmbz" podUID="f75e11d0-c0de-4a8d-849d-807a87aa1497"
Jul 07 11:27:17.080164 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:17.079750    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dljdk" event={"ID":"b2009767-f15b-464f-8a3f-87335106fe11","Type":"ContainerStarted","Data":"ade9f6f9989097e8f7253203ffa522a1ef367ece4bc86f186a0dfc8659e616d8"}
Jul 07 11:27:17.081160 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:17.081139    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lmxhq" event={"ID":"ea85da56-a626-4fc7-a0d4-31b668987496","Type":"ContainerStarted","Data":"cfe6d1f5badcc058af1cfba6ef7b914e0bafdae3a9f6016675b696169f2f9920"}
Jul 07 11:27:17.082390 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:17.082368    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qh2dm" event={"ID":"e54ef86d-1806-4e84-a235-aba912f88f58","Type":"ContainerStarted","Data":"2856503e75b142a929c01c86204bdcfd6c2ead6296e8af00dea59626f8a67132"}
Jul 07 11:27:17.083705 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:17.083680    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lj4rs" event={"ID":"4e27418c-5ace-4a75-a4b9-931fffa64650","Type":"ContainerStarted","Data":"08624164aa2fb584faf938cf6d1c0ce223892b4ce2c8e208d812a145b86632ff"}
Jul 07 11:27:17.085118 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:17.085089    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rkn2t" event={"ID":"650c2a15-194e-47f4-99c8-9f94802d1e9b","Type":"ContainerStarted","Data":"0cdb105390319599e7a57faa062e342c12f3554fe862896ff0ff4d454d21745f"}
Jul 07 11:27:17.086402 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:17.086382    2570 generic.go:356] "Generic (PLEG): container finished" podID="20c2dc90-b7d3-4add-af1f-31f1ba991c54" containerID="5ef0faa347ede9729110ac3dddbc07516fc6433cb4023bbc4d2af1714c437b44" exitCode=0
Jul 07 11:27:17.086499 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:17.086460    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8f25d" event={"ID":"20c2dc90-b7d3-4add-af1f-31f1ba991c54","Type":"ContainerDied","Data":"5ef0faa347ede9729110ac3dddbc07516fc6433cb4023bbc4d2af1714c437b44"}
Jul 07 11:27:17.092811 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:17.092736    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw" event={"ID":"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b","Type":"ContainerStarted","Data":"f37c8a751f7e9d1e3d5b5ff9d77167c0fc1d2ad6b97fc114d21f274c28e0abdd"}
Jul 07 11:27:17.092892 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:17.092856    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw" event={"ID":"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b","Type":"ContainerStarted","Data":"c70f61463f7a97c20885b66d5a4709f7ea353e7bf3af7b0036e4d62bcd0454be"}
Jul 07 11:27:17.092935 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:17.092892    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw" event={"ID":"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b","Type":"ContainerStarted","Data":"aa9f5ccef2310ac193d27d308a79b3ffd16795f200b601f89a51af58465ebc0f"}
Jul 07 11:27:17.092935 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:17.092920    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw" event={"ID":"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b","Type":"ContainerStarted","Data":"50e4b32158981406f2677e4fd34a772192b8e318f503c1d58bddb384d80fd87c"}
Jul 07 11:27:17.093008 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:17.092964    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw" event={"ID":"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b","Type":"ContainerStarted","Data":"2fd4c688268805bfbdd3436edec03533dcadaa9ddf0f24295f7725dc067c8b1d"}
Jul 07 11:27:17.094074 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:17.094053    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7tk6n" event={"ID":"af613b4a-9c2e-4b9b-8a10-0ca67f1f9159","Type":"ContainerStarted","Data":"666c72793cef217c81c64c09a83755a4fd72c1895e730c85ba6d76eb7cf5395d"}
Jul 07 11:27:17.098595 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:17.098561    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-dljdk" podStartSLOduration=2.800960859 podStartE2EDuration="22.098549549s" totalImagesPullingTime="19.29758869s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:26:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:27:17.098366537 +0000 UTC m=+25.072006948" watchObservedRunningTime="2026-07-07 11:27:17.098549549 +0000 UTC m=+25.072189974"
Jul 07 11:27:17.098691 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:17.098672    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-78.ec2.internal" podStartSLOduration=22.098667603 podStartE2EDuration="22.098667603s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:26:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:26:59.066697506 +0000 UTC m=+7.040337936" watchObservedRunningTime="2026-07-07 11:27:17.098667603 +0000 UTC m=+25.072308016"
Jul 07 11:27:17.114935 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:17.114896    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-7tk6n" podStartSLOduration=12.272471985 podStartE2EDuration="23.11488077s" totalImagesPullingTime="10.842408785s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:26:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:27:17.114325732 +0000 UTC m=+25.087966142" watchObservedRunningTime="2026-07-07 11:27:17.11488077 +0000 UTC m=+25.088521180"
Jul 07 11:27:17.179310 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:17.179249    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qh2dm" podStartSLOduration=3.904062625 podStartE2EDuration="23.179226628s" totalImagesPullingTime="19.275164003s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:26:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:27:17.154355102 +0000 UTC m=+25.127995511" watchObservedRunningTime="2026-07-07 11:27:17.179226628 +0000 UTC m=+25.152867033"
Jul 07 11:27:17.179764 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:17.179729    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-lj4rs" podStartSLOduration=3.848956561 podStartE2EDuration="23.179721072s" totalImagesPullingTime="19.330764511s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:26:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:27:17.179141902 +0000 UTC m=+25.152782313" watchObservedRunningTime="2026-07-07 11:27:17.179721072 +0000 UTC m=+25.153361486"
Jul 07 11:27:17.225701 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:17.225652    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-rkn2t" podStartSLOduration=3.9458299119999998 podStartE2EDuration="23.22563609s" totalImagesPullingTime="19.279806178s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:26:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:27:17.22557211 +0000 UTC m=+25.199212520" watchObservedRunningTime="2026-07-07 11:27:17.22563609 +0000 UTC m=+25.199276500"
Jul 07 11:27:17.932723 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:17.932225    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8f5jf"
Jul 07 11:27:17.932723 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:17.932269    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7bk7n"
Jul 07 11:27:17.932723 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:17.932239    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ppmbz"
Jul 07 11:27:17.932723 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:17.932362    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8f5jf" podUID="afdb1ba0-560a-4b0f-883b-aa960c6dd304"
Jul 07 11:27:17.932723 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:17.932438    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ppmbz" podUID="f75e11d0-c0de-4a8d-849d-807a87aa1497"
Jul 07 11:27:17.932723 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:17.932514    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7bk7n" podUID="732e808f-49d9-44a1-b93f-1cdb95c31e0b"
Jul 07 11:27:18.019363 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:18.019339    2570 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 07 11:27:18.099730 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:18.099639    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw" event={"ID":"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b","Type":"ContainerStarted","Data":"95b5364078f1d25278e5222bb721bd18cb2c908b17907b093c918cd9f9703170"}
Jul 07 11:27:18.101575 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:18.101548    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lmxhq" event={"ID":"ea85da56-a626-4fc7-a0d4-31b668987496","Type":"ContainerStarted","Data":"4960f7ebb037fd0bcfe5ae55ff3588b8a2538c21341aec6217cf00f0d50d4a83"}
Jul 07 11:27:18.103031 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:18.103006    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vdrl5" event={"ID":"b9392160-8536-4d5e-916d-e2a0164db043","Type":"ContainerStarted","Data":"2c4c27af18fd14241ad4243cf258b2e41bf5119ab33d603c483bc1966a831a22"}
Jul 07 11:27:18.133999 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:18.133953    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-vdrl5" podStartSLOduration=4.846676363 podStartE2EDuration="24.133930747s" totalImagesPullingTime="19.287254384s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:26:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:27:18.133848813 +0000 UTC m=+26.107489215" watchObservedRunningTime="2026-07-07 11:27:18.133930747 +0000 UTC m=+26.107571159"
Jul 07 11:27:18.911307 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:18.911026    2570 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T11:27:18.019357276Z","UUID":"737246a0-ee17-42b2-ac98-85839622ef9f","Handler":null,"Name":"","Endpoint":""}
Jul 07 11:27:18.915098 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:18.913450    2570 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 07 11:27:18.915098 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:18.913499    2570 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 07 11:27:19.931893 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:19.931857    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8f5jf"
Jul 07 11:27:19.932485 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:19.931857    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7bk7n"
Jul 07 11:27:19.932485 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:19.931999    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8f5jf" podUID="afdb1ba0-560a-4b0f-883b-aa960c6dd304"
Jul 07 11:27:19.932485 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:19.932098    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7bk7n" podUID="732e808f-49d9-44a1-b93f-1cdb95c31e0b"
Jul 07 11:27:19.932485 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:19.931859    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ppmbz"
Jul 07 11:27:19.932485 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:19.932230    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ppmbz" podUID="f75e11d0-c0de-4a8d-849d-807a87aa1497"
Jul 07 11:27:20.110031 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:20.109994    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw" event={"ID":"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b","Type":"ContainerStarted","Data":"73ad32d7c07aa1e17bf20408368bb6ff04057dc7795e9c2ac32903c80d339c50"}
Jul 07 11:27:20.111790 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:20.111765    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lmxhq" event={"ID":"ea85da56-a626-4fc7-a0d4-31b668987496","Type":"ContainerStarted","Data":"5a9e2355ec363c8efdac4781c269807316d0fd797bc1cf544fbd120b7cce953c"}
Jul 07 11:27:20.131321 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:20.131269    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lmxhq" podStartSLOduration=4.6496411779999995 podStartE2EDuration="26.131252834s" totalImagesPullingTime="21.481611656s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:26:54 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:27:20.130905158 +0000 UTC m=+28.104545581" watchObservedRunningTime="2026-07-07 11:27:20.131252834 +0000 UTC m=+28.104893245"
Jul 07 11:27:21.865118 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:21.865076    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-7tk6n"
Jul 07 11:27:21.865945 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:21.865924    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-7tk6n"
Jul 07 11:27:21.932593 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:21.932522    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ppmbz"
Jul 07 11:27:21.932695 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:21.932664    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ppmbz" podUID="f75e11d0-c0de-4a8d-849d-807a87aa1497"
Jul 07 11:27:21.932815 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:21.932749    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7bk7n"
Jul 07 11:27:21.932876 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:21.932757    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8f5jf"
Jul 07 11:27:21.932876 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:21.932835    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7bk7n" podUID="732e808f-49d9-44a1-b93f-1cdb95c31e0b"
Jul 07 11:27:21.932982 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:21.932932    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8f5jf" podUID="afdb1ba0-560a-4b0f-883b-aa960c6dd304"
Jul 07 11:27:22.121324 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:22.120988    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw" event={"ID":"6400d4cc-3b3b-4fd0-aa78-a8a3e007a36b","Type":"ContainerStarted","Data":"90c2812b32d53de958027588b413b56dbc0c610d3945b5233cc5adb5b8555d5f"}
Jul 07 11:27:22.121502 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:22.121336    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:27:22.121502 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:22.121387    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:27:22.123237 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:22.123186    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8f25d" event={"ID":"20c2dc90-b7d3-4add-af1f-31f1ba991c54","Type":"ContainerStarted","Data":"695e810869b251f8c4ec33a1031a602ac96c7a05e3a761d86ddee44321a62f0b"}
Jul 07 11:27:22.123608 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:22.123591    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-7tk6n"
Jul 07 11:27:22.124200 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:22.124180    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-7tk6n"
Jul 07 11:27:22.140556 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:22.140533    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:27:22.149114 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:22.149074    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw" podStartSLOduration=8.806385001 podStartE2EDuration="28.149060708s" totalImagesPullingTime="19.342675707s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:26:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:27:22.148678332 +0000 UTC m=+30.122318742" watchObservedRunningTime="2026-07-07 11:27:22.149060708 +0000 UTC m=+30.122701118"
Jul 07 11:27:23.127244 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:23.127197    2570 generic.go:356] "Generic (PLEG): container finished" podID="20c2dc90-b7d3-4add-af1f-31f1ba991c54" containerID="695e810869b251f8c4ec33a1031a602ac96c7a05e3a761d86ddee44321a62f0b" exitCode=0
Jul 07 11:27:23.127758 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:23.127243    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8f25d" event={"ID":"20c2dc90-b7d3-4add-af1f-31f1ba991c54","Type":"ContainerDied","Data":"695e810869b251f8c4ec33a1031a602ac96c7a05e3a761d86ddee44321a62f0b"}
Jul 07 11:27:23.127758 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:23.127671    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:27:23.144472 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:23.144448    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:27:23.933297 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:23.933262    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8f5jf"
Jul 07 11:27:23.933562 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:23.933414    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8f5jf" podUID="afdb1ba0-560a-4b0f-883b-aa960c6dd304"
Jul 07 11:27:23.933562 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:23.933527    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7bk7n"
Jul 07 11:27:23.933688 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:23.933620    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7bk7n" podUID="732e808f-49d9-44a1-b93f-1cdb95c31e0b"
Jul 07 11:27:23.933688 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:23.933684    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ppmbz"
Jul 07 11:27:23.933798 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:23.933767    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ppmbz" podUID="f75e11d0-c0de-4a8d-849d-807a87aa1497"
Jul 07 11:27:23.935641 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:23.935612    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7bk7n"]
Jul 07 11:27:23.936657 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:23.936632    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8f5jf"]
Jul 07 11:27:23.937552 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:23.937526    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ppmbz"]
Jul 07 11:27:24.129599 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:24.129575    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ppmbz"
Jul 07 11:27:24.129958 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:24.129636    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7bk7n"
Jul 07 11:27:24.129958 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:24.129820    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7bk7n" podUID="732e808f-49d9-44a1-b93f-1cdb95c31e0b"
Jul 07 11:27:24.129958 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:24.129837    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8f5jf"
Jul 07 11:27:24.129958 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:24.129907    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8f5jf" podUID="afdb1ba0-560a-4b0f-883b-aa960c6dd304"
Jul 07 11:27:24.130106 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:24.129981    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ppmbz" podUID="f75e11d0-c0de-4a8d-849d-807a87aa1497"
Jul 07 11:27:25.134009 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:25.133973    2570 generic.go:356] "Generic (PLEG): container finished" podID="20c2dc90-b7d3-4add-af1f-31f1ba991c54" containerID="8f2b56166a88ecfe3371652a48dc1760c5f6965ca30ebd5fe4a422301f883d5b" exitCode=0
Jul 07 11:27:25.134396 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:25.134049    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8f25d" event={"ID":"20c2dc90-b7d3-4add-af1f-31f1ba991c54","Type":"ContainerDied","Data":"8f2b56166a88ecfe3371652a48dc1760c5f6965ca30ebd5fe4a422301f883d5b"}
Jul 07 11:27:25.932581 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:25.932546    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8f5jf"
Jul 07 11:27:25.932756 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:25.932546    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ppmbz"
Jul 07 11:27:25.932756 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:25.932681    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7bk7n"
Jul 07 11:27:25.932756 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:25.932679    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8f5jf" podUID="afdb1ba0-560a-4b0f-883b-aa960c6dd304"
Jul 07 11:27:25.932917 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:25.932833    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ppmbz" podUID="f75e11d0-c0de-4a8d-849d-807a87aa1497"
Jul 07 11:27:25.932975 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:25.932914    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7bk7n" podUID="732e808f-49d9-44a1-b93f-1cdb95c31e0b"
Jul 07 11:27:27.140424 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:27.140389    2570 generic.go:356] "Generic (PLEG): container finished" podID="20c2dc90-b7d3-4add-af1f-31f1ba991c54" containerID="a1041c5e38b63b367b41b58b7e1e5b27557864854d3a7e9ef1dfad47497ca8e6" exitCode=0
Jul 07 11:27:27.141201 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:27.140437    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8f25d" event={"ID":"20c2dc90-b7d3-4add-af1f-31f1ba991c54","Type":"ContainerDied","Data":"a1041c5e38b63b367b41b58b7e1e5b27557864854d3a7e9ef1dfad47497ca8e6"}
Jul 07 11:27:27.692681 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:27.692643    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f75e11d0-c0de-4a8d-849d-807a87aa1497-metrics-certs\") pod \"network-metrics-daemon-ppmbz\" (UID: \"f75e11d0-c0de-4a8d-849d-807a87aa1497\") " pod="openshift-multus/network-metrics-daemon-ppmbz"
Jul 07 11:27:27.692858 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:27.692701    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/afdb1ba0-560a-4b0f-883b-aa960c6dd304-original-pull-secret\") pod \"global-pull-secret-syncer-8f5jf\" (UID: \"afdb1ba0-560a-4b0f-883b-aa960c6dd304\") " pod="kube-system/global-pull-secret-syncer-8f5jf"
Jul 07 11:27:27.692858 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:27.692795    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 11:27:27.692858 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:27.692798    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 11:27:27.692858 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:27.692848    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afdb1ba0-560a-4b0f-883b-aa960c6dd304-original-pull-secret podName:afdb1ba0-560a-4b0f-883b-aa960c6dd304 nodeName:}" failed. No retries permitted until 2026-07-07 11:27:59.692834653 +0000 UTC m=+67.666475041 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/afdb1ba0-560a-4b0f-883b-aa960c6dd304-original-pull-secret") pod "global-pull-secret-syncer-8f5jf" (UID: "afdb1ba0-560a-4b0f-883b-aa960c6dd304") : object "kube-system"/"original-pull-secret" not registered
Jul 07 11:27:27.692988 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:27.692862    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f75e11d0-c0de-4a8d-849d-807a87aa1497-metrics-certs podName:f75e11d0-c0de-4a8d-849d-807a87aa1497 nodeName:}" failed. No retries permitted until 2026-07-07 11:27:59.692855494 +0000 UTC m=+67.666495881 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f75e11d0-c0de-4a8d-849d-807a87aa1497-metrics-certs") pod "network-metrics-daemon-ppmbz" (UID: "f75e11d0-c0de-4a8d-849d-807a87aa1497") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 11:27:27.793359 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:27.793321    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hb26c\" (UniqueName: \"kubernetes.io/projected/732e808f-49d9-44a1-b93f-1cdb95c31e0b-kube-api-access-hb26c\") pod \"network-check-target-7bk7n\" (UID: \"732e808f-49d9-44a1-b93f-1cdb95c31e0b\") " pod="openshift-network-diagnostics/network-check-target-7bk7n"
Jul 07 11:27:27.793534 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:27.793460    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 11:27:27.793534 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:27.793478    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 11:27:27.793534 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:27.793487    2570 projected.go:196] Error preparing data for projected volume kube-api-access-hb26c for pod openshift-network-diagnostics/network-check-target-7bk7n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 11:27:27.793534 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:27.793534    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/732e808f-49d9-44a1-b93f-1cdb95c31e0b-kube-api-access-hb26c podName:732e808f-49d9-44a1-b93f-1cdb95c31e0b nodeName:}" failed. No retries permitted until 2026-07-07 11:27:59.793521274 +0000 UTC m=+67.767161663 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-hb26c" (UniqueName: "kubernetes.io/projected/732e808f-49d9-44a1-b93f-1cdb95c31e0b-kube-api-access-hb26c") pod "network-check-target-7bk7n" (UID: "732e808f-49d9-44a1-b93f-1cdb95c31e0b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 11:27:27.932225 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:27.932181    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8f5jf"
Jul 07 11:27:27.932398 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:27.932186    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ppmbz"
Jul 07 11:27:27.932398 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:27.932328    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8f5jf" podUID="afdb1ba0-560a-4b0f-883b-aa960c6dd304"
Jul 07 11:27:27.932398 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:27.932189    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7bk7n"
Jul 07 11:27:27.932532 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:27.932390    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ppmbz" podUID="f75e11d0-c0de-4a8d-849d-807a87aa1497"
Jul 07 11:27:27.932532 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:27.932464    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7bk7n" podUID="732e808f-49d9-44a1-b93f-1cdb95c31e0b"
Jul 07 11:27:29.244383 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.244303    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-78.ec2.internal" event="NodeReady"
Jul 07 11:27:29.244821 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.244421    2570 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 07 11:27:29.284092 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.284056    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-696d746fbb-8plv4"]
Jul 07 11:27:29.288062 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.288028    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-wlk99"]
Jul 07 11:27:29.288249 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.288150    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-696d746fbb-8plv4"
Jul 07 11:27:29.291317 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.291100    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 07 11:27:29.291445 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.291328    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-7m7fm\""
Jul 07 11:27:29.291445 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.291202    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 07 11:27:29.291522 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.291196    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 07 11:27:29.292112 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.292090    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-wlk99"
Jul 07 11:27:29.295071 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.295014    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 07 11:27:29.295071 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.295045    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 07 11:27:29.295588 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.295567    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-vgmpj\""
Jul 07 11:27:29.300323 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.300301    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 07 11:27:29.303876 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.303614    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-696d746fbb-8plv4"]
Jul 07 11:27:29.304999 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.304226    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-wlk99"]
Jul 07 11:27:29.313695 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.313670    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-9gqzt"]
Jul 07 11:27:29.317513 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.317492    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-zd288"]
Jul 07 11:27:29.317644 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.317610    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9gqzt"
Jul 07 11:27:29.321057 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.321035    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zd288"
Jul 07 11:27:29.321167 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.321048    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 07 11:27:29.321465 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.321445    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 07 11:27:29.321704 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.321685    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xqnk4\""
Jul 07 11:27:29.324004 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.323985    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-7kkm5\""
Jul 07 11:27:29.324272 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.324244    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 07 11:27:29.324365 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.324255    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 07 11:27:29.324431 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.324359    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 07 11:27:29.329787 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.329678    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9gqzt"]
Jul 07 11:27:29.330858 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.330836    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zd288"]
Jul 07 11:27:29.405586 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.405549    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/88a181af-4d7c-4398-a53b-69ab8f2e4035-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-wlk99\" (UID: \"88a181af-4d7c-4398-a53b-69ab8f2e4035\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-wlk99"
Jul 07 11:27:29.405586 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.405594    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkfjz\" (UniqueName: \"kubernetes.io/projected/24d1f3b4-0970-41a7-95e6-6ab8e00f5b47-kube-api-access-dkfjz\") pod \"ingress-canary-zd288\" (UID: \"24d1f3b4-0970-41a7-95e6-6ab8e00f5b47\") " pod="openshift-ingress-canary/ingress-canary-zd288"
Jul 07 11:27:29.405833 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.405616    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1aa5a3e9-5a49-4837-8fd2-389682ccef44-image-registry-private-configuration\") pod \"image-registry-696d746fbb-8plv4\" (UID: \"1aa5a3e9-5a49-4837-8fd2-389682ccef44\") " pod="openshift-image-registry/image-registry-696d746fbb-8plv4"
Jul 07 11:27:29.405833 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.405709    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/24d1f3b4-0970-41a7-95e6-6ab8e00f5b47-cert\") pod \"ingress-canary-zd288\" (UID: \"24d1f3b4-0970-41a7-95e6-6ab8e00f5b47\") " pod="openshift-ingress-canary/ingress-canary-zd288"
Jul 07 11:27:29.405833 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.405809    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1aa5a3e9-5a49-4837-8fd2-389682ccef44-trusted-ca\") pod \"image-registry-696d746fbb-8plv4\" (UID: \"1aa5a3e9-5a49-4837-8fd2-389682ccef44\") " pod="openshift-image-registry/image-registry-696d746fbb-8plv4"
Jul 07 11:27:29.405942 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.405845    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1aa5a3e9-5a49-4837-8fd2-389682ccef44-registry-certificates\") pod \"image-registry-696d746fbb-8plv4\" (UID: \"1aa5a3e9-5a49-4837-8fd2-389682ccef44\") " pod="openshift-image-registry/image-registry-696d746fbb-8plv4"
Jul 07 11:27:29.405942 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.405913    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/88a181af-4d7c-4398-a53b-69ab8f2e4035-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-wlk99\" (UID: \"88a181af-4d7c-4398-a53b-69ab8f2e4035\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-wlk99"
Jul 07 11:27:29.405942 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.405938    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bfcc5fcf-866f-405b-ae62-dcf401627bc7-config-volume\") pod \"dns-default-9gqzt\" (UID: \"bfcc5fcf-866f-405b-ae62-dcf401627bc7\") " pod="openshift-dns/dns-default-9gqzt"
Jul 07 11:27:29.406066 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.405964    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1aa5a3e9-5a49-4837-8fd2-389682ccef44-registry-tls\") pod \"image-registry-696d746fbb-8plv4\" (UID: \"1aa5a3e9-5a49-4837-8fd2-389682ccef44\") " pod="openshift-image-registry/image-registry-696d746fbb-8plv4"
Jul 07 11:27:29.406066 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.405990    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1aa5a3e9-5a49-4837-8fd2-389682ccef44-installation-pull-secrets\") pod \"image-registry-696d746fbb-8plv4\" (UID: \"1aa5a3e9-5a49-4837-8fd2-389682ccef44\") " pod="openshift-image-registry/image-registry-696d746fbb-8plv4"
Jul 07 11:27:29.406066 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.406030    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7v7sh\" (UniqueName: \"kubernetes.io/projected/1aa5a3e9-5a49-4837-8fd2-389682ccef44-kube-api-access-7v7sh\") pod \"image-registry-696d746fbb-8plv4\" (UID: \"1aa5a3e9-5a49-4837-8fd2-389682ccef44\") " pod="openshift-image-registry/image-registry-696d746fbb-8plv4"
Jul 07 11:27:29.406232 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.406127    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1aa5a3e9-5a49-4837-8fd2-389682ccef44-bound-sa-token\") pod \"image-registry-696d746fbb-8plv4\" (UID: \"1aa5a3e9-5a49-4837-8fd2-389682ccef44\") " pod="openshift-image-registry/image-registry-696d746fbb-8plv4"
Jul 07 11:27:29.406232 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.406167    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bfcc5fcf-866f-405b-ae62-dcf401627bc7-tmp-dir\") pod \"dns-default-9gqzt\" (UID: \"bfcc5fcf-866f-405b-ae62-dcf401627bc7\") " pod="openshift-dns/dns-default-9gqzt"
Jul 07 11:27:29.406232 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.406194    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kllfz\" (UniqueName: \"kubernetes.io/projected/bfcc5fcf-866f-405b-ae62-dcf401627bc7-kube-api-access-kllfz\") pod \"dns-default-9gqzt\" (UID: \"bfcc5fcf-866f-405b-ae62-dcf401627bc7\") " pod="openshift-dns/dns-default-9gqzt"
Jul 07 11:27:29.406359 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.406247    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bfcc5fcf-866f-405b-ae62-dcf401627bc7-metrics-tls\") pod \"dns-default-9gqzt\" (UID: \"bfcc5fcf-866f-405b-ae62-dcf401627bc7\") " pod="openshift-dns/dns-default-9gqzt"
Jul 07 11:27:29.406359 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.406273    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1aa5a3e9-5a49-4837-8fd2-389682ccef44-ca-trust-extracted\") pod \"image-registry-696d746fbb-8plv4\" (UID: \"1aa5a3e9-5a49-4837-8fd2-389682ccef44\") " pod="openshift-image-registry/image-registry-696d746fbb-8plv4"
Jul 07 11:27:29.507733 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.507648    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1aa5a3e9-5a49-4837-8fd2-389682ccef44-trusted-ca\") pod \"image-registry-696d746fbb-8plv4\" (UID: \"1aa5a3e9-5a49-4837-8fd2-389682ccef44\") " pod="openshift-image-registry/image-registry-696d746fbb-8plv4"
Jul 07 11:27:29.507733 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.507687    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1aa5a3e9-5a49-4837-8fd2-389682ccef44-registry-certificates\") pod \"image-registry-696d746fbb-8plv4\" (UID: \"1aa5a3e9-5a49-4837-8fd2-389682ccef44\") " pod="openshift-image-registry/image-registry-696d746fbb-8plv4"
Jul 07 11:27:29.507733 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.507718    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/88a181af-4d7c-4398-a53b-69ab8f2e4035-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-wlk99\" (UID: \"88a181af-4d7c-4398-a53b-69ab8f2e4035\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-wlk99"
Jul 07 11:27:29.508001 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.507741    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bfcc5fcf-866f-405b-ae62-dcf401627bc7-config-volume\") pod \"dns-default-9gqzt\" (UID: \"bfcc5fcf-866f-405b-ae62-dcf401627bc7\") " pod="openshift-dns/dns-default-9gqzt"
Jul 07 11:27:29.508001 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.507765    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1aa5a3e9-5a49-4837-8fd2-389682ccef44-registry-tls\") pod \"image-registry-696d746fbb-8plv4\" (UID: \"1aa5a3e9-5a49-4837-8fd2-389682ccef44\") " pod="openshift-image-registry/image-registry-696d746fbb-8plv4"
Jul 07 11:27:29.508001 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.507788    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1aa5a3e9-5a49-4837-8fd2-389682ccef44-installation-pull-secrets\") pod \"image-registry-696d746fbb-8plv4\" (UID: \"1aa5a3e9-5a49-4837-8fd2-389682ccef44\") " pod="openshift-image-registry/image-registry-696d746fbb-8plv4"
Jul 07 11:27:29.508001 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.507824    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7v7sh\" (UniqueName: \"kubernetes.io/projected/1aa5a3e9-5a49-4837-8fd2-389682ccef44-kube-api-access-7v7sh\") pod \"image-registry-696d746fbb-8plv4\" (UID: \"1aa5a3e9-5a49-4837-8fd2-389682ccef44\") " pod="openshift-image-registry/image-registry-696d746fbb-8plv4"
Jul 07 11:27:29.508001 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.507874    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1aa5a3e9-5a49-4837-8fd2-389682ccef44-bound-sa-token\") pod \"image-registry-696d746fbb-8plv4\" (UID: \"1aa5a3e9-5a49-4837-8fd2-389682ccef44\") " pod="openshift-image-registry/image-registry-696d746fbb-8plv4"
Jul 07 11:27:29.508001 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.507928    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bfcc5fcf-866f-405b-ae62-dcf401627bc7-tmp-dir\") pod \"dns-default-9gqzt\" (UID: \"bfcc5fcf-866f-405b-ae62-dcf401627bc7\") " pod="openshift-dns/dns-default-9gqzt"
Jul 07 11:27:29.508001 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.507949    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kllfz\" (UniqueName: \"kubernetes.io/projected/bfcc5fcf-866f-405b-ae62-dcf401627bc7-kube-api-access-kllfz\") pod \"dns-default-9gqzt\" (UID: \"bfcc5fcf-866f-405b-ae62-dcf401627bc7\") " pod="openshift-dns/dns-default-9gqzt"
Jul 07 11:27:29.508001 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.507977    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bfcc5fcf-866f-405b-ae62-dcf401627bc7-metrics-tls\") pod \"dns-default-9gqzt\" (UID: \"bfcc5fcf-866f-405b-ae62-dcf401627bc7\") " pod="openshift-dns/dns-default-9gqzt"
Jul 07 11:27:29.508001 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.507999    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1aa5a3e9-5a49-4837-8fd2-389682ccef44-ca-trust-extracted\") pod \"image-registry-696d746fbb-8plv4\" (UID: \"1aa5a3e9-5a49-4837-8fd2-389682ccef44\") " pod="openshift-image-registry/image-registry-696d746fbb-8plv4"
Jul 07 11:27:29.508467 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.508037    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/88a181af-4d7c-4398-a53b-69ab8f2e4035-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-wlk99\" (UID: \"88a181af-4d7c-4398-a53b-69ab8f2e4035\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-wlk99"
Jul 07 11:27:29.508467 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.508069    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkfjz\" (UniqueName: \"kubernetes.io/projected/24d1f3b4-0970-41a7-95e6-6ab8e00f5b47-kube-api-access-dkfjz\") pod \"ingress-canary-zd288\" (UID: \"24d1f3b4-0970-41a7-95e6-6ab8e00f5b47\") " pod="openshift-ingress-canary/ingress-canary-zd288"
Jul 07 11:27:29.508467 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.508100    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1aa5a3e9-5a49-4837-8fd2-389682ccef44-image-registry-private-configuration\") pod \"image-registry-696d746fbb-8plv4\" (UID: \"1aa5a3e9-5a49-4837-8fd2-389682ccef44\") " pod="openshift-image-registry/image-registry-696d746fbb-8plv4"
Jul 07 11:27:29.508467 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.508163    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/24d1f3b4-0970-41a7-95e6-6ab8e00f5b47-cert\") pod \"ingress-canary-zd288\" (UID: \"24d1f3b4-0970-41a7-95e6-6ab8e00f5b47\") " pod="openshift-ingress-canary/ingress-canary-zd288"
Jul 07 11:27:29.508467 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:29.508231    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 11:27:29.508467 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:29.508306    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfcc5fcf-866f-405b-ae62-dcf401627bc7-metrics-tls podName:bfcc5fcf-866f-405b-ae62-dcf401627bc7 nodeName:}" failed. No retries permitted until 2026-07-07 11:27:30.008287003 +0000 UTC m=+37.981927413 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bfcc5fcf-866f-405b-ae62-dcf401627bc7-metrics-tls") pod "dns-default-9gqzt" (UID: "bfcc5fcf-866f-405b-ae62-dcf401627bc7") : secret "dns-default-metrics-tls" not found
Jul 07 11:27:29.508467 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.508361    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bfcc5fcf-866f-405b-ae62-dcf401627bc7-tmp-dir\") pod \"dns-default-9gqzt\" (UID: \"bfcc5fcf-866f-405b-ae62-dcf401627bc7\") " pod="openshift-dns/dns-default-9gqzt"
Jul 07 11:27:29.508467 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.508407    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1aa5a3e9-5a49-4837-8fd2-389682ccef44-registry-certificates\") pod \"image-registry-696d746fbb-8plv4\" (UID: \"1aa5a3e9-5a49-4837-8fd2-389682ccef44\") " pod="openshift-image-registry/image-registry-696d746fbb-8plv4"
Jul 07 11:27:29.508467 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.508414    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bfcc5fcf-866f-405b-ae62-dcf401627bc7-config-volume\") pod \"dns-default-9gqzt\" (UID: \"bfcc5fcf-866f-405b-ae62-dcf401627bc7\") " pod="openshift-dns/dns-default-9gqzt"
Jul 07 11:27:29.508467 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:29.508459    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 11:27:29.508467 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:29.508473    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-696d746fbb-8plv4: secret "image-registry-tls" not found
Jul 07 11:27:29.508789 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:29.508502    2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 07 11:27:29.508789 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:29.508538    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1aa5a3e9-5a49-4837-8fd2-389682ccef44-registry-tls podName:1aa5a3e9-5a49-4837-8fd2-389682ccef44 nodeName:}" failed. No retries permitted until 2026-07-07 11:27:30.008513761 +0000 UTC m=+37.982154163 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1aa5a3e9-5a49-4837-8fd2-389682ccef44-registry-tls") pod "image-registry-696d746fbb-8plv4" (UID: "1aa5a3e9-5a49-4837-8fd2-389682ccef44") : secret "image-registry-tls" not found
Jul 07 11:27:29.508789 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:29.508563    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88a181af-4d7c-4398-a53b-69ab8f2e4035-networking-console-plugin-cert podName:88a181af-4d7c-4398-a53b-69ab8f2e4035 nodeName:}" failed. No retries permitted until 2026-07-07 11:27:30.008548568 +0000 UTC m=+37.982188972 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/88a181af-4d7c-4398-a53b-69ab8f2e4035-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-wlk99" (UID: "88a181af-4d7c-4398-a53b-69ab8f2e4035") : secret "networking-console-plugin-cert" not found
Jul 07 11:27:29.508789 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.508713    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1aa5a3e9-5a49-4837-8fd2-389682ccef44-ca-trust-extracted\") pod \"image-registry-696d746fbb-8plv4\" (UID: \"1aa5a3e9-5a49-4837-8fd2-389682ccef44\") " pod="openshift-image-registry/image-registry-696d746fbb-8plv4"
Jul 07 11:27:29.508951 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.508878    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1aa5a3e9-5a49-4837-8fd2-389682ccef44-trusted-ca\") pod \"image-registry-696d746fbb-8plv4\" (UID: \"1aa5a3e9-5a49-4837-8fd2-389682ccef44\") " pod="openshift-image-registry/image-registry-696d746fbb-8plv4"
Jul 07 11:27:29.509381 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.509352    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/88a181af-4d7c-4398-a53b-69ab8f2e4035-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-wlk99\" (UID: \"88a181af-4d7c-4398-a53b-69ab8f2e4035\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-wlk99"
Jul 07 11:27:29.513243 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.513196    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1aa5a3e9-5a49-4837-8fd2-389682ccef44-installation-pull-secrets\") pod \"image-registry-696d746fbb-8plv4\" (UID: \"1aa5a3e9-5a49-4837-8fd2-389682ccef44\") " pod="openshift-image-registry/image-registry-696d746fbb-8plv4"
Jul 07 11:27:29.513348 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.513266    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1aa5a3e9-5a49-4837-8fd2-389682ccef44-image-registry-private-configuration\") pod \"image-registry-696d746fbb-8plv4\" (UID: \"1aa5a3e9-5a49-4837-8fd2-389682ccef44\") " pod="openshift-image-registry/image-registry-696d746fbb-8plv4"
Jul 07 11:27:29.513348 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.513289    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/24d1f3b4-0970-41a7-95e6-6ab8e00f5b47-cert\") pod \"ingress-canary-zd288\" (UID: \"24d1f3b4-0970-41a7-95e6-6ab8e00f5b47\") " pod="openshift-ingress-canary/ingress-canary-zd288"
Jul 07 11:27:29.517710 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.517686    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkfjz\" (UniqueName: \"kubernetes.io/projected/24d1f3b4-0970-41a7-95e6-6ab8e00f5b47-kube-api-access-dkfjz\") pod \"ingress-canary-zd288\" (UID: \"24d1f3b4-0970-41a7-95e6-6ab8e00f5b47\") " pod="openshift-ingress-canary/ingress-canary-zd288"
Jul 07 11:27:29.521181 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.521151    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1aa5a3e9-5a49-4837-8fd2-389682ccef44-bound-sa-token\") pod \"image-registry-696d746fbb-8plv4\" (UID: \"1aa5a3e9-5a49-4837-8fd2-389682ccef44\") " pod="openshift-image-registry/image-registry-696d746fbb-8plv4"
Jul 07 11:27:29.521299 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.521225    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7v7sh\" (UniqueName: \"kubernetes.io/projected/1aa5a3e9-5a49-4837-8fd2-389682ccef44-kube-api-access-7v7sh\") pod \"image-registry-696d746fbb-8plv4\" (UID: \"1aa5a3e9-5a49-4837-8fd2-389682ccef44\") " pod="openshift-image-registry/image-registry-696d746fbb-8plv4"
Jul 07 11:27:29.522265 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.522242    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kllfz\" (UniqueName: \"kubernetes.io/projected/bfcc5fcf-866f-405b-ae62-dcf401627bc7-kube-api-access-kllfz\") pod \"dns-default-9gqzt\" (UID: \"bfcc5fcf-866f-405b-ae62-dcf401627bc7\") " pod="openshift-dns/dns-default-9gqzt"
Jul 07 11:27:29.635266 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.635191    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zd288"
Jul 07 11:27:29.782980 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.782861    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zd288"]
Jul 07 11:27:29.788076 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:27:29.788049    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24d1f3b4_0970_41a7_95e6_6ab8e00f5b47.slice/crio-9dc3f963e28484a06993fdbca91e843533ce5f18dfdb1379abc087ab4d2280a7 WatchSource:0}: Error finding container 9dc3f963e28484a06993fdbca91e843533ce5f18dfdb1379abc087ab4d2280a7: Status 404 returned error can't find the container with id 9dc3f963e28484a06993fdbca91e843533ce5f18dfdb1379abc087ab4d2280a7
Jul 07 11:27:29.932204 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.932166    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7bk7n"
Jul 07 11:27:29.932377 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.932166    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8f5jf"
Jul 07 11:27:29.932377 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.932329    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ppmbz"
Jul 07 11:27:29.935683 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.935443    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 07 11:27:29.935683 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.935453    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 07 11:27:29.936726 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.936704    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-nfmsb\""
Jul 07 11:27:29.936726 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.936718    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-xcgtk\""
Jul 07 11:27:29.936726 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.936734    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 07 11:27:29.936969 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:29.936769    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 07 11:27:30.013063 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:30.013025    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bfcc5fcf-866f-405b-ae62-dcf401627bc7-metrics-tls\") pod \"dns-default-9gqzt\" (UID: \"bfcc5fcf-866f-405b-ae62-dcf401627bc7\") " pod="openshift-dns/dns-default-9gqzt"
Jul 07 11:27:30.013063 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:30.013071    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/88a181af-4d7c-4398-a53b-69ab8f2e4035-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-wlk99\" (UID: \"88a181af-4d7c-4398-a53b-69ab8f2e4035\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-wlk99"
Jul 07 11:27:30.013346 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:30.013166    2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 07 11:27:30.013346 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:30.013182    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 11:27:30.013346 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:30.013260    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfcc5fcf-866f-405b-ae62-dcf401627bc7-metrics-tls podName:bfcc5fcf-866f-405b-ae62-dcf401627bc7 nodeName:}" failed. No retries permitted until 2026-07-07 11:27:31.013242865 +0000 UTC m=+38.986883253 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bfcc5fcf-866f-405b-ae62-dcf401627bc7-metrics-tls") pod "dns-default-9gqzt" (UID: "bfcc5fcf-866f-405b-ae62-dcf401627bc7") : secret "dns-default-metrics-tls" not found
Jul 07 11:27:30.013346 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:30.013328    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1aa5a3e9-5a49-4837-8fd2-389682ccef44-registry-tls\") pod \"image-registry-696d746fbb-8plv4\" (UID: \"1aa5a3e9-5a49-4837-8fd2-389682ccef44\") " pod="openshift-image-registry/image-registry-696d746fbb-8plv4"
Jul 07 11:27:30.013346 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:30.013336    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88a181af-4d7c-4398-a53b-69ab8f2e4035-networking-console-plugin-cert podName:88a181af-4d7c-4398-a53b-69ab8f2e4035 nodeName:}" failed. No retries permitted until 2026-07-07 11:27:31.013319854 +0000 UTC m=+38.986960257 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/88a181af-4d7c-4398-a53b-69ab8f2e4035-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-wlk99" (UID: "88a181af-4d7c-4398-a53b-69ab8f2e4035") : secret "networking-console-plugin-cert" not found
Jul 07 11:27:30.013582 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:30.013403    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 11:27:30.013582 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:30.013419    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-696d746fbb-8plv4: secret "image-registry-tls" not found
Jul 07 11:27:30.013582 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:30.013461    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1aa5a3e9-5a49-4837-8fd2-389682ccef44-registry-tls podName:1aa5a3e9-5a49-4837-8fd2-389682ccef44 nodeName:}" failed. No retries permitted until 2026-07-07 11:27:31.013448668 +0000 UTC m=+38.987089058 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1aa5a3e9-5a49-4837-8fd2-389682ccef44-registry-tls") pod "image-registry-696d746fbb-8plv4" (UID: "1aa5a3e9-5a49-4837-8fd2-389682ccef44") : secret "image-registry-tls" not found
Jul 07 11:27:30.147897 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:30.147817    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zd288" event={"ID":"24d1f3b4-0970-41a7-95e6-6ab8e00f5b47","Type":"ContainerStarted","Data":"9dc3f963e28484a06993fdbca91e843533ce5f18dfdb1379abc087ab4d2280a7"}
Jul 07 11:27:31.023270 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:31.023236    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1aa5a3e9-5a49-4837-8fd2-389682ccef44-registry-tls\") pod \"image-registry-696d746fbb-8plv4\" (UID: \"1aa5a3e9-5a49-4837-8fd2-389682ccef44\") " pod="openshift-image-registry/image-registry-696d746fbb-8plv4"
Jul 07 11:27:31.023814 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:31.023301    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bfcc5fcf-866f-405b-ae62-dcf401627bc7-metrics-tls\") pod \"dns-default-9gqzt\" (UID: \"bfcc5fcf-866f-405b-ae62-dcf401627bc7\") " pod="openshift-dns/dns-default-9gqzt"
Jul 07 11:27:31.023814 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:31.023340    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/88a181af-4d7c-4398-a53b-69ab8f2e4035-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-wlk99\" (UID: \"88a181af-4d7c-4398-a53b-69ab8f2e4035\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-wlk99"
Jul 07 11:27:31.023814 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:31.023397    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 11:27:31.023814 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:31.023419    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-696d746fbb-8plv4: secret "image-registry-tls" not found
Jul 07 11:27:31.023814 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:31.023454    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 11:27:31.023814 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:31.023486    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1aa5a3e9-5a49-4837-8fd2-389682ccef44-registry-tls podName:1aa5a3e9-5a49-4837-8fd2-389682ccef44 nodeName:}" failed. No retries permitted until 2026-07-07 11:27:33.023466787 +0000 UTC m=+40.997107194 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1aa5a3e9-5a49-4837-8fd2-389682ccef44-registry-tls") pod "image-registry-696d746fbb-8plv4" (UID: "1aa5a3e9-5a49-4837-8fd2-389682ccef44") : secret "image-registry-tls" not found
Jul 07 11:27:31.023814 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:31.023487    2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 07 11:27:31.023814 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:31.023504    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfcc5fcf-866f-405b-ae62-dcf401627bc7-metrics-tls podName:bfcc5fcf-866f-405b-ae62-dcf401627bc7 nodeName:}" failed. No retries permitted until 2026-07-07 11:27:33.023494874 +0000 UTC m=+40.997135268 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bfcc5fcf-866f-405b-ae62-dcf401627bc7-metrics-tls") pod "dns-default-9gqzt" (UID: "bfcc5fcf-866f-405b-ae62-dcf401627bc7") : secret "dns-default-metrics-tls" not found
Jul 07 11:27:31.023814 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:31.023543    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88a181af-4d7c-4398-a53b-69ab8f2e4035-networking-console-plugin-cert podName:88a181af-4d7c-4398-a53b-69ab8f2e4035 nodeName:}" failed. No retries permitted until 2026-07-07 11:27:33.023528069 +0000 UTC m=+40.997168475 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/88a181af-4d7c-4398-a53b-69ab8f2e4035-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-wlk99" (UID: "88a181af-4d7c-4398-a53b-69ab8f2e4035") : secret "networking-console-plugin-cert" not found
Jul 07 11:27:33.042007 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:33.041966    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/88a181af-4d7c-4398-a53b-69ab8f2e4035-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-wlk99\" (UID: \"88a181af-4d7c-4398-a53b-69ab8f2e4035\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-wlk99"
Jul 07 11:27:33.042404 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:33.042079    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1aa5a3e9-5a49-4837-8fd2-389682ccef44-registry-tls\") pod \"image-registry-696d746fbb-8plv4\" (UID: \"1aa5a3e9-5a49-4837-8fd2-389682ccef44\") " pod="openshift-image-registry/image-registry-696d746fbb-8plv4"
Jul 07 11:27:33.042404 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:33.042115    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bfcc5fcf-866f-405b-ae62-dcf401627bc7-metrics-tls\") pod \"dns-default-9gqzt\" (UID: \"bfcc5fcf-866f-405b-ae62-dcf401627bc7\") " pod="openshift-dns/dns-default-9gqzt"
Jul 07 11:27:33.042404 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:33.042152    2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 07 11:27:33.042404 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:33.042203    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 11:27:33.042404 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:33.042232    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 11:27:33.042404 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:33.042249    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-696d746fbb-8plv4: secret "image-registry-tls" not found
Jul 07 11:27:33.042404 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:33.042258    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88a181af-4d7c-4398-a53b-69ab8f2e4035-networking-console-plugin-cert podName:88a181af-4d7c-4398-a53b-69ab8f2e4035 nodeName:}" failed. No retries permitted until 2026-07-07 11:27:37.042235644 +0000 UTC m=+45.015876050 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/88a181af-4d7c-4398-a53b-69ab8f2e4035-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-wlk99" (UID: "88a181af-4d7c-4398-a53b-69ab8f2e4035") : secret "networking-console-plugin-cert" not found
Jul 07 11:27:33.042404 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:33.042278    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfcc5fcf-866f-405b-ae62-dcf401627bc7-metrics-tls podName:bfcc5fcf-866f-405b-ae62-dcf401627bc7 nodeName:}" failed. No retries permitted until 2026-07-07 11:27:37.04226802 +0000 UTC m=+45.015908407 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bfcc5fcf-866f-405b-ae62-dcf401627bc7-metrics-tls") pod "dns-default-9gqzt" (UID: "bfcc5fcf-866f-405b-ae62-dcf401627bc7") : secret "dns-default-metrics-tls" not found
Jul 07 11:27:33.042404 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:33.042290    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1aa5a3e9-5a49-4837-8fd2-389682ccef44-registry-tls podName:1aa5a3e9-5a49-4837-8fd2-389682ccef44 nodeName:}" failed. No retries permitted until 2026-07-07 11:27:37.042284121 +0000 UTC m=+45.015924509 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1aa5a3e9-5a49-4837-8fd2-389682ccef44-registry-tls") pod "image-registry-696d746fbb-8plv4" (UID: "1aa5a3e9-5a49-4837-8fd2-389682ccef44") : secret "image-registry-tls" not found
Jul 07 11:27:34.157840 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:34.157611    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zd288" event={"ID":"24d1f3b4-0970-41a7-95e6-6ab8e00f5b47","Type":"ContainerStarted","Data":"86f2a7d9fbff973ba44905d313f1a1dc03ecd80108ff69aae231dc5b45d3d65f"}
Jul 07 11:27:34.160424 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:34.160399    2570 generic.go:356] "Generic (PLEG): container finished" podID="20c2dc90-b7d3-4add-af1f-31f1ba991c54" containerID="c82f6ab0eaf561c8e222d4c6250dc3b044e1fd147701a610e4f766c4422cbf42" exitCode=0
Jul 07 11:27:34.160594 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:34.160436    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8f25d" event={"ID":"20c2dc90-b7d3-4add-af1f-31f1ba991c54","Type":"ContainerDied","Data":"c82f6ab0eaf561c8e222d4c6250dc3b044e1fd147701a610e4f766c4422cbf42"}
Jul 07 11:27:34.173412 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:34.173372    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-zd288" podStartSLOduration=1.8111973959999998 podStartE2EDuration="5.173357988s" totalImagesPullingTime="3.362160592s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:27:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:27:34.172675799 +0000 UTC m=+42.146316211" watchObservedRunningTime="2026-07-07 11:27:34.173357988 +0000 UTC m=+42.146998398"
Jul 07 11:27:35.165542 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:35.165508    2570 generic.go:356] "Generic (PLEG): container finished" podID="20c2dc90-b7d3-4add-af1f-31f1ba991c54" containerID="db30c9324941b50c8d89209954fc9fc48b9d02f8690eed21074fb34fcf7d3200" exitCode=0
Jul 07 11:27:35.165991 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:35.165596    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8f25d" event={"ID":"20c2dc90-b7d3-4add-af1f-31f1ba991c54","Type":"ContainerDied","Data":"db30c9324941b50c8d89209954fc9fc48b9d02f8690eed21074fb34fcf7d3200"}
Jul 07 11:27:36.171019 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:36.170985    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8f25d" event={"ID":"20c2dc90-b7d3-4add-af1f-31f1ba991c54","Type":"ContainerStarted","Data":"e55ba6508ef374f4a6611a0d5b512264774e801137aa40806a515704a512e6fa"}
Jul 07 11:27:36.196448 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:36.196400    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-8f25d" podStartSLOduration=10.092858319 podStartE2EDuration="42.196384137s" totalImagesPullingTime="32.103525818s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:26:54 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:27:36.194351826 +0000 UTC m=+44.167992236" watchObservedRunningTime="2026-07-07 11:27:36.196384137 +0000 UTC m=+44.170024562"
Jul 07 11:27:37.075065 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:37.075022    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1aa5a3e9-5a49-4837-8fd2-389682ccef44-registry-tls\") pod \"image-registry-696d746fbb-8plv4\" (UID: \"1aa5a3e9-5a49-4837-8fd2-389682ccef44\") " pod="openshift-image-registry/image-registry-696d746fbb-8plv4"
Jul 07 11:27:37.075281 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:37.075083    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bfcc5fcf-866f-405b-ae62-dcf401627bc7-metrics-tls\") pod \"dns-default-9gqzt\" (UID: \"bfcc5fcf-866f-405b-ae62-dcf401627bc7\") " pod="openshift-dns/dns-default-9gqzt"
Jul 07 11:27:37.075281 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:37.075117    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/88a181af-4d7c-4398-a53b-69ab8f2e4035-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-wlk99\" (UID: \"88a181af-4d7c-4398-a53b-69ab8f2e4035\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-wlk99"
Jul 07 11:27:37.075281 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:37.075172    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 11:27:37.075281 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:37.075195    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-696d746fbb-8plv4: secret "image-registry-tls" not found
Jul 07 11:27:37.075281 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:37.075234    2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 07 11:27:37.075281 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:37.075175    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 11:27:37.075281 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:37.075271    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1aa5a3e9-5a49-4837-8fd2-389682ccef44-registry-tls podName:1aa5a3e9-5a49-4837-8fd2-389682ccef44 nodeName:}" failed. No retries permitted until 2026-07-07 11:27:45.075250744 +0000 UTC m=+53.048891151 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1aa5a3e9-5a49-4837-8fd2-389682ccef44-registry-tls") pod "image-registry-696d746fbb-8plv4" (UID: "1aa5a3e9-5a49-4837-8fd2-389682ccef44") : secret "image-registry-tls" not found
Jul 07 11:27:37.075281 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:37.075287    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88a181af-4d7c-4398-a53b-69ab8f2e4035-networking-console-plugin-cert podName:88a181af-4d7c-4398-a53b-69ab8f2e4035 nodeName:}" failed. No retries permitted until 2026-07-07 11:27:45.075280758 +0000 UTC m=+53.048921145 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/88a181af-4d7c-4398-a53b-69ab8f2e4035-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-wlk99" (UID: "88a181af-4d7c-4398-a53b-69ab8f2e4035") : secret "networking-console-plugin-cert" not found
Jul 07 11:27:37.075549 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:37.075309    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfcc5fcf-866f-405b-ae62-dcf401627bc7-metrics-tls podName:bfcc5fcf-866f-405b-ae62-dcf401627bc7 nodeName:}" failed. No retries permitted until 2026-07-07 11:27:45.075295137 +0000 UTC m=+53.048935525 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bfcc5fcf-866f-405b-ae62-dcf401627bc7-metrics-tls") pod "dns-default-9gqzt" (UID: "bfcc5fcf-866f-405b-ae62-dcf401627bc7") : secret "dns-default-metrics-tls" not found
Jul 07 11:27:45.136703 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:45.136663    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1aa5a3e9-5a49-4837-8fd2-389682ccef44-registry-tls\") pod \"image-registry-696d746fbb-8plv4\" (UID: \"1aa5a3e9-5a49-4837-8fd2-389682ccef44\") " pod="openshift-image-registry/image-registry-696d746fbb-8plv4"
Jul 07 11:27:45.137268 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:45.136717    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bfcc5fcf-866f-405b-ae62-dcf401627bc7-metrics-tls\") pod \"dns-default-9gqzt\" (UID: \"bfcc5fcf-866f-405b-ae62-dcf401627bc7\") " pod="openshift-dns/dns-default-9gqzt"
Jul 07 11:27:45.137268 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:45.136741    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/88a181af-4d7c-4398-a53b-69ab8f2e4035-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-wlk99\" (UID: \"88a181af-4d7c-4398-a53b-69ab8f2e4035\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-wlk99"
Jul 07 11:27:45.137268 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:45.136820    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 11:27:45.137268 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:45.136841    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-696d746fbb-8plv4: secret "image-registry-tls" not found
Jul 07 11:27:45.137268 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:45.136839    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 11:27:45.137268 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:45.136850    2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 07 11:27:45.137268 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:45.136892    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1aa5a3e9-5a49-4837-8fd2-389682ccef44-registry-tls podName:1aa5a3e9-5a49-4837-8fd2-389682ccef44 nodeName:}" failed. No retries permitted until 2026-07-07 11:28:01.13687553 +0000 UTC m=+69.110515918 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1aa5a3e9-5a49-4837-8fd2-389682ccef44-registry-tls") pod "image-registry-696d746fbb-8plv4" (UID: "1aa5a3e9-5a49-4837-8fd2-389682ccef44") : secret "image-registry-tls" not found
Jul 07 11:27:45.137268 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:45.136943    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfcc5fcf-866f-405b-ae62-dcf401627bc7-metrics-tls podName:bfcc5fcf-866f-405b-ae62-dcf401627bc7 nodeName:}" failed. No retries permitted until 2026-07-07 11:28:01.136926154 +0000 UTC m=+69.110566541 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bfcc5fcf-866f-405b-ae62-dcf401627bc7-metrics-tls") pod "dns-default-9gqzt" (UID: "bfcc5fcf-866f-405b-ae62-dcf401627bc7") : secret "dns-default-metrics-tls" not found
Jul 07 11:27:45.137268 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:45.136967    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88a181af-4d7c-4398-a53b-69ab8f2e4035-networking-console-plugin-cert podName:88a181af-4d7c-4398-a53b-69ab8f2e4035 nodeName:}" failed. No retries permitted until 2026-07-07 11:28:01.136956814 +0000 UTC m=+69.110597206 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/88a181af-4d7c-4398-a53b-69ab8f2e4035-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-wlk99" (UID: "88a181af-4d7c-4398-a53b-69ab8f2e4035") : secret "networking-console-plugin-cert" not found
Jul 07 11:27:55.154330 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:55.154300    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pm4tw"
Jul 07 11:27:59.742413 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:59.742353    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/afdb1ba0-560a-4b0f-883b-aa960c6dd304-original-pull-secret\") pod \"global-pull-secret-syncer-8f5jf\" (UID: \"afdb1ba0-560a-4b0f-883b-aa960c6dd304\") " pod="kube-system/global-pull-secret-syncer-8f5jf"
Jul 07 11:27:59.742849 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:59.742460    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f75e11d0-c0de-4a8d-849d-807a87aa1497-metrics-certs\") pod \"network-metrics-daemon-ppmbz\" (UID: \"f75e11d0-c0de-4a8d-849d-807a87aa1497\") " pod="openshift-multus/network-metrics-daemon-ppmbz"
Jul 07 11:27:59.745256 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:59.745233    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 07 11:27:59.745341 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:59.745268    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 07 11:27:59.753461 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:59.753433    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 07 11:27:59.753559 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:27:59.753509    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f75e11d0-c0de-4a8d-849d-807a87aa1497-metrics-certs podName:f75e11d0-c0de-4a8d-849d-807a87aa1497 nodeName:}" failed. No retries permitted until 2026-07-07 11:29:03.753492846 +0000 UTC m=+131.727133234 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f75e11d0-c0de-4a8d-849d-807a87aa1497-metrics-certs") pod "network-metrics-daemon-ppmbz" (UID: "f75e11d0-c0de-4a8d-849d-807a87aa1497") : secret "metrics-daemon-secret" not found
Jul 07 11:27:59.756630 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:59.756603    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/afdb1ba0-560a-4b0f-883b-aa960c6dd304-original-pull-secret\") pod \"global-pull-secret-syncer-8f5jf\" (UID: \"afdb1ba0-560a-4b0f-883b-aa960c6dd304\") " pod="kube-system/global-pull-secret-syncer-8f5jf"
Jul 07 11:27:59.843845 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:59.843805    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hb26c\" (UniqueName: \"kubernetes.io/projected/732e808f-49d9-44a1-b93f-1cdb95c31e0b-kube-api-access-hb26c\") pod \"network-check-target-7bk7n\" (UID: \"732e808f-49d9-44a1-b93f-1cdb95c31e0b\") " pod="openshift-network-diagnostics/network-check-target-7bk7n"
Jul 07 11:27:59.846834 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:59.846815    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 07 11:27:59.856968 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:59.856943    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 07 11:27:59.867625 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:59.867600    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hb26c\" (UniqueName: \"kubernetes.io/projected/732e808f-49d9-44a1-b93f-1cdb95c31e0b-kube-api-access-hb26c\") pod \"network-check-target-7bk7n\" (UID: \"732e808f-49d9-44a1-b93f-1cdb95c31e0b\") " pod="openshift-network-diagnostics/network-check-target-7bk7n"
Jul 07 11:27:59.947726 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:59.947698    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-xcgtk\""
Jul 07 11:27:59.952950 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:59.952934    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8f5jf"
Jul 07 11:27:59.955061 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:27:59.955040    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7bk7n"
Jul 07 11:28:00.101367 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:28:00.101338    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8f5jf"]
Jul 07 11:28:00.104453 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:28:00.104430    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafdb1ba0_560a_4b0f_883b_aa960c6dd304.slice/crio-a10763a7f39172dae4127926781285f1fae47239c7b615570668df5044a3b6ad WatchSource:0}: Error finding container a10763a7f39172dae4127926781285f1fae47239c7b615570668df5044a3b6ad: Status 404 returned error can't find the container with id a10763a7f39172dae4127926781285f1fae47239c7b615570668df5044a3b6ad
Jul 07 11:28:00.113768 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:28:00.113745    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7bk7n"]
Jul 07 11:28:00.116145 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:28:00.116126    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod732e808f_49d9_44a1_b93f_1cdb95c31e0b.slice/crio-02b5b37840f6d73a864140d6fe4aae8d0603ef346be4355fc38baa17aa37a288 WatchSource:0}: Error finding container 02b5b37840f6d73a864140d6fe4aae8d0603ef346be4355fc38baa17aa37a288: Status 404 returned error can't find the container with id 02b5b37840f6d73a864140d6fe4aae8d0603ef346be4355fc38baa17aa37a288
Jul 07 11:28:00.229107 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:28:00.229064    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8f5jf" event={"ID":"afdb1ba0-560a-4b0f-883b-aa960c6dd304","Type":"ContainerStarted","Data":"a10763a7f39172dae4127926781285f1fae47239c7b615570668df5044a3b6ad"}
Jul 07 11:28:00.230112 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:28:00.230088    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7bk7n" event={"ID":"732e808f-49d9-44a1-b93f-1cdb95c31e0b","Type":"ContainerStarted","Data":"02b5b37840f6d73a864140d6fe4aae8d0603ef346be4355fc38baa17aa37a288"}
Jul 07 11:28:01.154832 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:28:01.154749    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bfcc5fcf-866f-405b-ae62-dcf401627bc7-metrics-tls\") pod \"dns-default-9gqzt\" (UID: \"bfcc5fcf-866f-405b-ae62-dcf401627bc7\") " pod="openshift-dns/dns-default-9gqzt"
Jul 07 11:28:01.154832 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:28:01.154810    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/88a181af-4d7c-4398-a53b-69ab8f2e4035-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-wlk99\" (UID: \"88a181af-4d7c-4398-a53b-69ab8f2e4035\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-wlk99"
Jul 07 11:28:01.155396 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:28:01.154885    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1aa5a3e9-5a49-4837-8fd2-389682ccef44-registry-tls\") pod \"image-registry-696d746fbb-8plv4\" (UID: \"1aa5a3e9-5a49-4837-8fd2-389682ccef44\") " pod="openshift-image-registry/image-registry-696d746fbb-8plv4"
Jul 07 11:28:01.155396 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:28:01.154904    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 11:28:01.155396 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:28:01.154983    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfcc5fcf-866f-405b-ae62-dcf401627bc7-metrics-tls podName:bfcc5fcf-866f-405b-ae62-dcf401627bc7 nodeName:}" failed. No retries permitted until 2026-07-07 11:28:33.154962373 +0000 UTC m=+101.128602764 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bfcc5fcf-866f-405b-ae62-dcf401627bc7-metrics-tls") pod "dns-default-9gqzt" (UID: "bfcc5fcf-866f-405b-ae62-dcf401627bc7") : secret "dns-default-metrics-tls" not found
Jul 07 11:28:01.155396 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:28:01.155019    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 11:28:01.155396 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:28:01.155034    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-696d746fbb-8plv4: secret "image-registry-tls" not found
Jul 07 11:28:01.155396 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:28:01.155082    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1aa5a3e9-5a49-4837-8fd2-389682ccef44-registry-tls podName:1aa5a3e9-5a49-4837-8fd2-389682ccef44 nodeName:}" failed. No retries permitted until 2026-07-07 11:28:33.155065693 +0000 UTC m=+101.128706094 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1aa5a3e9-5a49-4837-8fd2-389682ccef44-registry-tls") pod "image-registry-696d746fbb-8plv4" (UID: "1aa5a3e9-5a49-4837-8fd2-389682ccef44") : secret "image-registry-tls" not found
Jul 07 11:28:01.155396 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:28:01.155150    2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 07 11:28:01.155396 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:28:01.155185    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88a181af-4d7c-4398-a53b-69ab8f2e4035-networking-console-plugin-cert podName:88a181af-4d7c-4398-a53b-69ab8f2e4035 nodeName:}" failed. No retries permitted until 2026-07-07 11:28:33.155173803 +0000 UTC m=+101.128814192 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/88a181af-4d7c-4398-a53b-69ab8f2e4035-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-wlk99" (UID: "88a181af-4d7c-4398-a53b-69ab8f2e4035") : secret "networking-console-plugin-cert" not found
Jul 07 11:28:05.244124 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:28:05.244081    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8f5jf" event={"ID":"afdb1ba0-560a-4b0f-883b-aa960c6dd304","Type":"ContainerStarted","Data":"785dcb2ee010f8faf3b13f8f08321d55e4d7c2fd6d69d5f8fc8956ea26cb57bf"}
Jul 07 11:28:05.245522 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:28:05.245494    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7bk7n" event={"ID":"732e808f-49d9-44a1-b93f-1cdb95c31e0b","Type":"ContainerStarted","Data":"8c2036e1a11869a2ff4a5151dae4bbbef68fc461d4f6ddb67314f30b0e161859"}
Jul 07 11:28:05.245628 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:28:05.245616    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-7bk7n"
Jul 07 11:28:05.261272 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:28:05.261205    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-8f5jf" podStartSLOduration=66.648645299 podStartE2EDuration="1m11.261190064s" totalImagesPullingTime="4.612544765s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:26:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:28:05.260151949 +0000 UTC m=+73.233792359" watchObservedRunningTime="2026-07-07 11:28:05.261190064 +0000 UTC m=+73.234830474"
Jul 07 11:28:05.276858 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:28:05.276809    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-7bk7n" podStartSLOduration=66.680829119 podStartE2EDuration="1m11.276792121s" totalImagesPullingTime="4.595963002s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:26:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:28:05.276318992 +0000 UTC m=+73.249959402" watchObservedRunningTime="2026-07-07 11:28:05.276792121 +0000 UTC m=+73.250432530"
Jul 07 11:28:33.183203 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:28:33.183068    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1aa5a3e9-5a49-4837-8fd2-389682ccef44-registry-tls\") pod \"image-registry-696d746fbb-8plv4\" (UID: \"1aa5a3e9-5a49-4837-8fd2-389682ccef44\") " pod="openshift-image-registry/image-registry-696d746fbb-8plv4"
Jul 07 11:28:33.183203 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:28:33.183134    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bfcc5fcf-866f-405b-ae62-dcf401627bc7-metrics-tls\") pod \"dns-default-9gqzt\" (UID: \"bfcc5fcf-866f-405b-ae62-dcf401627bc7\") " pod="openshift-dns/dns-default-9gqzt"
Jul 07 11:28:33.183203 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:28:33.183166    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/88a181af-4d7c-4398-a53b-69ab8f2e4035-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-wlk99\" (UID: \"88a181af-4d7c-4398-a53b-69ab8f2e4035\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-wlk99"
Jul 07 11:28:33.183779 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:28:33.183234    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 11:28:33.183779 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:28:33.183254    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-696d746fbb-8plv4: secret "image-registry-tls" not found
Jul 07 11:28:33.183779 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:28:33.183290    2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 07 11:28:33.183779 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:28:33.183309    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1aa5a3e9-5a49-4837-8fd2-389682ccef44-registry-tls podName:1aa5a3e9-5a49-4837-8fd2-389682ccef44 nodeName:}" failed. No retries permitted until 2026-07-07 11:29:37.183294039 +0000 UTC m=+165.156934428 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1aa5a3e9-5a49-4837-8fd2-389682ccef44-registry-tls") pod "image-registry-696d746fbb-8plv4" (UID: "1aa5a3e9-5a49-4837-8fd2-389682ccef44") : secret "image-registry-tls" not found
Jul 07 11:28:33.183779 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:28:33.183356    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88a181af-4d7c-4398-a53b-69ab8f2e4035-networking-console-plugin-cert podName:88a181af-4d7c-4398-a53b-69ab8f2e4035 nodeName:}" failed. No retries permitted until 2026-07-07 11:29:37.183337566 +0000 UTC m=+165.156977970 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/88a181af-4d7c-4398-a53b-69ab8f2e4035-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-wlk99" (UID: "88a181af-4d7c-4398-a53b-69ab8f2e4035") : secret "networking-console-plugin-cert" not found
Jul 07 11:28:33.183779 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:28:33.183291    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 11:28:33.183779 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:28:33.183418    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfcc5fcf-866f-405b-ae62-dcf401627bc7-metrics-tls podName:bfcc5fcf-866f-405b-ae62-dcf401627bc7 nodeName:}" failed. No retries permitted until 2026-07-07 11:29:37.183405416 +0000 UTC m=+165.157045804 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bfcc5fcf-866f-405b-ae62-dcf401627bc7-metrics-tls") pod "dns-default-9gqzt" (UID: "bfcc5fcf-866f-405b-ae62-dcf401627bc7") : secret "dns-default-metrics-tls" not found
Jul 07 11:28:36.250354 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:28:36.250323    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-7bk7n"
Jul 07 11:28:38.448394 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:28:38.448359    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-549f87767d-p27zg"]
Jul 07 11:28:38.451032 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:28:38.451015    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-549f87767d-p27zg"
Jul 07 11:28:38.453539 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:28:38.453511    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 07 11:28:38.453658 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:28:38.453584    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-k7zwv\""
Jul 07 11:28:38.454964 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:28:38.454945    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 07 11:28:38.455049 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:28:38.454991    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 07 11:28:38.455049 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:28:38.454997    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 07 11:28:38.461309 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:28:38.461287    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-549f87767d-p27zg"]
Jul 07 11:28:38.620618 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:28:38.620571    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f5615715-cede-47e4-bfdb-297722c0ec36-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-549f87767d-p27zg\" (UID: \"f5615715-cede-47e4-bfdb-297722c0ec36\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-549f87767d-p27zg"
Jul 07 11:28:38.620792 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:28:38.620647    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldlbz\" (UniqueName: \"kubernetes.io/projected/f5615715-cede-47e4-bfdb-297722c0ec36-kube-api-access-ldlbz\") pod \"managed-serviceaccount-addon-agent-549f87767d-p27zg\" (UID: \"f5615715-cede-47e4-bfdb-297722c0ec36\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-549f87767d-p27zg"
Jul 07 11:28:38.721229 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:28:38.721132    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f5615715-cede-47e4-bfdb-297722c0ec36-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-549f87767d-p27zg\" (UID: \"f5615715-cede-47e4-bfdb-297722c0ec36\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-549f87767d-p27zg"
Jul 07 11:28:38.721317 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:28:38.721232    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldlbz\" (UniqueName: \"kubernetes.io/projected/f5615715-cede-47e4-bfdb-297722c0ec36-kube-api-access-ldlbz\") pod \"managed-serviceaccount-addon-agent-549f87767d-p27zg\" (UID: \"f5615715-cede-47e4-bfdb-297722c0ec36\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-549f87767d-p27zg"
Jul 07 11:28:38.723645 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:28:38.723615    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f5615715-cede-47e4-bfdb-297722c0ec36-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-549f87767d-p27zg\" (UID: \"f5615715-cede-47e4-bfdb-297722c0ec36\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-549f87767d-p27zg"
Jul 07 11:28:38.730053 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:28:38.730025    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldlbz\" (UniqueName: \"kubernetes.io/projected/f5615715-cede-47e4-bfdb-297722c0ec36-kube-api-access-ldlbz\") pod \"managed-serviceaccount-addon-agent-549f87767d-p27zg\" (UID: \"f5615715-cede-47e4-bfdb-297722c0ec36\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-549f87767d-p27zg"
Jul 07 11:28:38.773314 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:28:38.773284    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-549f87767d-p27zg"
Jul 07 11:28:38.894391 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:28:38.894350    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-549f87767d-p27zg"]
Jul 07 11:28:38.898164 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:28:38.898138    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf5615715_cede_47e4_bfdb_297722c0ec36.slice/crio-94d3b7d3a1fb9c5a04994885fce8ad892690fd5223772d03bf555da34bf58ceb WatchSource:0}: Error finding container 94d3b7d3a1fb9c5a04994885fce8ad892690fd5223772d03bf555da34bf58ceb: Status 404 returned error can't find the container with id 94d3b7d3a1fb9c5a04994885fce8ad892690fd5223772d03bf555da34bf58ceb
Jul 07 11:28:39.324993 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:28:39.324956    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-549f87767d-p27zg" event={"ID":"f5615715-cede-47e4-bfdb-297722c0ec36","Type":"ContainerStarted","Data":"94d3b7d3a1fb9c5a04994885fce8ad892690fd5223772d03bf555da34bf58ceb"}
Jul 07 11:28:42.334007 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:28:42.333966    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-549f87767d-p27zg" event={"ID":"f5615715-cede-47e4-bfdb-297722c0ec36","Type":"ContainerStarted","Data":"2e6e1f4277662581db248ed763759c143d0b6933240b412f9b81f1b9c17ee485"}
Jul 07 11:28:42.351654 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:28:42.351608    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-549f87767d-p27zg" podStartSLOduration=1.3898870159999999 podStartE2EDuration="4.351591496s" totalImagesPullingTime="2.96170448s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:28:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:28:42.349894643 +0000 UTC m=+110.323535065" watchObservedRunningTime="2026-07-07 11:28:42.351591496 +0000 UTC m=+110.325231908"
Jul 07 11:29:03.806141 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:29:03.806091    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f75e11d0-c0de-4a8d-849d-807a87aa1497-metrics-certs\") pod \"network-metrics-daemon-ppmbz\" (UID: \"f75e11d0-c0de-4a8d-849d-807a87aa1497\") " pod="openshift-multus/network-metrics-daemon-ppmbz"
Jul 07 11:29:03.806598 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:29:03.806258    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 07 11:29:03.806598 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:29:03.806332    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f75e11d0-c0de-4a8d-849d-807a87aa1497-metrics-certs podName:f75e11d0-c0de-4a8d-849d-807a87aa1497 nodeName:}" failed. No retries permitted until 2026-07-07 11:31:05.806314926 +0000 UTC m=+253.779955314 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f75e11d0-c0de-4a8d-849d-807a87aa1497-metrics-certs") pod "network-metrics-daemon-ppmbz" (UID: "f75e11d0-c0de-4a8d-849d-807a87aa1497") : secret "metrics-daemon-secret" not found
Jul 07 11:29:32.300999 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:29:32.300953    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-696d746fbb-8plv4" podUID="1aa5a3e9-5a49-4837-8fd2-389682ccef44"
Jul 07 11:29:32.308124 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:29:32.308089    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6bffdf9778-wlk99" podUID="88a181af-4d7c-4398-a53b-69ab8f2e4035"
Jul 07 11:29:32.328446 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:29:32.328409    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-9gqzt" podUID="bfcc5fcf-866f-405b-ae62-dcf401627bc7"
Jul 07 11:29:32.456652 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:29:32.456625    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-696d746fbb-8plv4"
Jul 07 11:29:32.456813 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:29:32.456628    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9gqzt"
Jul 07 11:29:32.456813 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:29:32.456628    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-wlk99"
Jul 07 11:29:32.958651 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:29:32.958591    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-ppmbz" podUID="f75e11d0-c0de-4a8d-849d-807a87aa1497"
Jul 07 11:29:37.249421 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:29:37.249381    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1aa5a3e9-5a49-4837-8fd2-389682ccef44-registry-tls\") pod \"image-registry-696d746fbb-8plv4\" (UID: \"1aa5a3e9-5a49-4837-8fd2-389682ccef44\") " pod="openshift-image-registry/image-registry-696d746fbb-8plv4"
Jul 07 11:29:37.249829 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:29:37.249435    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bfcc5fcf-866f-405b-ae62-dcf401627bc7-metrics-tls\") pod \"dns-default-9gqzt\" (UID: \"bfcc5fcf-866f-405b-ae62-dcf401627bc7\") " pod="openshift-dns/dns-default-9gqzt"
Jul 07 11:29:37.249829 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:29:37.249461    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/88a181af-4d7c-4398-a53b-69ab8f2e4035-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-wlk99\" (UID: \"88a181af-4d7c-4398-a53b-69ab8f2e4035\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-wlk99"
Jul 07 11:29:37.249829 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:29:37.249545    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 11:29:37.249829 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:29:37.249561    2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 07 11:29:37.249829 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:29:37.249566    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-696d746fbb-8plv4: secret "image-registry-tls" not found
Jul 07 11:29:37.249829 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:29:37.249576    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 11:29:37.249829 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:29:37.249620    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1aa5a3e9-5a49-4837-8fd2-389682ccef44-registry-tls podName:1aa5a3e9-5a49-4837-8fd2-389682ccef44 nodeName:}" failed. No retries permitted until 2026-07-07 11:31:39.249604139 +0000 UTC m=+287.223244526 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1aa5a3e9-5a49-4837-8fd2-389682ccef44-registry-tls") pod "image-registry-696d746fbb-8plv4" (UID: "1aa5a3e9-5a49-4837-8fd2-389682ccef44") : secret "image-registry-tls" not found
Jul 07 11:29:37.249829 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:29:37.249634    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88a181af-4d7c-4398-a53b-69ab8f2e4035-networking-console-plugin-cert podName:88a181af-4d7c-4398-a53b-69ab8f2e4035 nodeName:}" failed. No retries permitted until 2026-07-07 11:31:39.249627622 +0000 UTC m=+287.223268010 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/88a181af-4d7c-4398-a53b-69ab8f2e4035-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-wlk99" (UID: "88a181af-4d7c-4398-a53b-69ab8f2e4035") : secret "networking-console-plugin-cert" not found
Jul 07 11:29:37.249829 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:29:37.249643    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfcc5fcf-866f-405b-ae62-dcf401627bc7-metrics-tls podName:bfcc5fcf-866f-405b-ae62-dcf401627bc7 nodeName:}" failed. No retries permitted until 2026-07-07 11:31:39.249638768 +0000 UTC m=+287.223279156 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bfcc5fcf-866f-405b-ae62-dcf401627bc7-metrics-tls") pod "dns-default-9gqzt" (UID: "bfcc5fcf-866f-405b-ae62-dcf401627bc7") : secret "dns-default-metrics-tls" not found
Jul 07 11:29:46.932744 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:29:46.932666    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ppmbz"
Jul 07 11:30:42.624559 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:30:42.624521    2570 generic.go:356] "Generic (PLEG): container finished" podID="f5615715-cede-47e4-bfdb-297722c0ec36" containerID="2e6e1f4277662581db248ed763759c143d0b6933240b412f9b81f1b9c17ee485" exitCode=255
Jul 07 11:30:42.624995 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:30:42.624583    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-549f87767d-p27zg" event={"ID":"f5615715-cede-47e4-bfdb-297722c0ec36","Type":"ContainerDied","Data":"2e6e1f4277662581db248ed763759c143d0b6933240b412f9b81f1b9c17ee485"}
Jul 07 11:30:42.630340 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:30:42.630321    2570 scope.go:117] "RemoveContainer" containerID="2e6e1f4277662581db248ed763759c143d0b6933240b412f9b81f1b9c17ee485"
Jul 07 11:30:43.628716 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:30:43.628677    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-549f87767d-p27zg" event={"ID":"f5615715-cede-47e4-bfdb-297722c0ec36","Type":"ContainerStarted","Data":"131947bb7d5a11c5c0919ba4cf14da2da91ff6a67852c4aebff9ae52eca9b33b"}
Jul 07 11:31:05.904952 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:05.904910    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f75e11d0-c0de-4a8d-849d-807a87aa1497-metrics-certs\") pod \"network-metrics-daemon-ppmbz\" (UID: \"f75e11d0-c0de-4a8d-849d-807a87aa1497\") " pod="openshift-multus/network-metrics-daemon-ppmbz"
Jul 07 11:31:05.905541 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:05.905077    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 07 11:31:05.905541 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:05.905192    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f75e11d0-c0de-4a8d-849d-807a87aa1497-metrics-certs podName:f75e11d0-c0de-4a8d-849d-807a87aa1497 nodeName:}" failed. No retries permitted until 2026-07-07 11:33:07.905170685 +0000 UTC m=+375.878811073 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f75e11d0-c0de-4a8d-849d-807a87aa1497-metrics-certs") pod "network-metrics-daemon-ppmbz" (UID: "f75e11d0-c0de-4a8d-849d-807a87aa1497") : secret "metrics-daemon-secret" not found
Jul 07 11:31:26.354044 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.353952    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-ktbpw"]
Jul 07 11:31:26.356017 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.355997    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-ktbpw"
Jul 07 11:31:26.359023 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.359000    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 07 11:31:26.359149 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.358998    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 07 11:31:26.359517 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.359500    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 07 11:31:26.360412 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.360395    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 07 11:31:26.360512 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.360396    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-x6g47\""
Jul 07 11:31:26.365732 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.365711    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 07 11:31:26.371393 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.371369    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-ktbpw"]
Jul 07 11:31:26.450955 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.450920    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ncgwv"]
Jul 07 11:31:26.452747 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.452725    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ncgwv"
Jul 07 11:31:26.455928 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.455901    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 07 11:31:26.456134 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.456115    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 07 11:31:26.456480 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.456458    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-mdkrb\""
Jul 07 11:31:26.456585 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.456537    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 07 11:31:26.457240 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.457203    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-ngjjc"]
Jul 07 11:31:26.457833 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.457816    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 07 11:31:26.459141 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.459122    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-5647bb9f4c-npcb5"]
Jul 07 11:31:26.459277 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.459260    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-ngjjc"
Jul 07 11:31:26.460895 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.460878    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5647bb9f4c-npcb5"
Jul 07 11:31:26.461148 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.461121    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/faf6f343-6515-4b2c-a34e-286cd2d9ec7a-config\") pod \"console-operator-68b757fb67-ktbpw\" (UID: \"faf6f343-6515-4b2c-a34e-286cd2d9ec7a\") " pod="openshift-console-operator/console-operator-68b757fb67-ktbpw"
Jul 07 11:31:26.461275 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.461191    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/faf6f343-6515-4b2c-a34e-286cd2d9ec7a-trusted-ca\") pod \"console-operator-68b757fb67-ktbpw\" (UID: \"faf6f343-6515-4b2c-a34e-286cd2d9ec7a\") " pod="openshift-console-operator/console-operator-68b757fb67-ktbpw"
Jul 07 11:31:26.461275 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.461263    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxz9q\" (UniqueName: \"kubernetes.io/projected/faf6f343-6515-4b2c-a34e-286cd2d9ec7a-kube-api-access-dxz9q\") pod \"console-operator-68b757fb67-ktbpw\" (UID: \"faf6f343-6515-4b2c-a34e-286cd2d9ec7a\") " pod="openshift-console-operator/console-operator-68b757fb67-ktbpw"
Jul 07 11:31:26.461400 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.461317    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/faf6f343-6515-4b2c-a34e-286cd2d9ec7a-serving-cert\") pod \"console-operator-68b757fb67-ktbpw\" (UID: \"faf6f343-6515-4b2c-a34e-286cd2d9ec7a\") " pod="openshift-console-operator/console-operator-68b757fb67-ktbpw"
Jul 07 11:31:26.461919 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.461901    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 07 11:31:26.462002 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.461904    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 07 11:31:26.462143 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.462129    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 07 11:31:26.462423 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.462406    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 07 11:31:26.463468 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.463453    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-f47b9\""
Jul 07 11:31:26.464081 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.463903    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 07 11:31:26.464081 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.463933    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-zshjf\""
Jul 07 11:31:26.464081 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.463945    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 07 11:31:26.464081 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.463967    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 07 11:31:26.464702 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.464686    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 07 11:31:26.464906 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.464891    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 07 11:31:26.464984 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.464944    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 07 11:31:26.470622 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.470582    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ncgwv"]
Jul 07 11:31:26.472588 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.472567    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-ngjjc"]
Jul 07 11:31:26.475472 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.475452    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 07 11:31:26.476115 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.476095    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5647bb9f4c-npcb5"]
Jul 07 11:31:26.562087 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.562048    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5742\" (UniqueName: \"kubernetes.io/projected/49b59d5c-4db1-4013-8d2e-ac8690c4723c-kube-api-access-x5742\") pod \"insights-operator-5c847dcfc4-ngjjc\" (UID: \"49b59d5c-4db1-4013-8d2e-ac8690c4723c\") " pod="openshift-insights/insights-operator-5c847dcfc4-ngjjc"
Jul 07 11:31:26.562087 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.562088    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/561f4302-647f-4dd2-bbcb-927a28c17df7-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-ncgwv\" (UID: \"561f4302-647f-4dd2-bbcb-927a28c17df7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ncgwv"
Jul 07 11:31:26.562374 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.562107    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/49b59d5c-4db1-4013-8d2e-ac8690c4723c-serving-cert\") pod \"insights-operator-5c847dcfc4-ngjjc\" (UID: \"49b59d5c-4db1-4013-8d2e-ac8690c4723c\") " pod="openshift-insights/insights-operator-5c847dcfc4-ngjjc"
Jul 07 11:31:26.562374 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.562183    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/331a680b-b6dd-46fd-b0da-cccfefd15bd3-metrics-certs\") pod \"router-default-5647bb9f4c-npcb5\" (UID: \"331a680b-b6dd-46fd-b0da-cccfefd15bd3\") " pod="openshift-ingress/router-default-5647bb9f4c-npcb5"
Jul 07 11:31:26.562374 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.562248    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/49b59d5c-4db1-4013-8d2e-ac8690c4723c-tmp\") pod \"insights-operator-5c847dcfc4-ngjjc\" (UID: \"49b59d5c-4db1-4013-8d2e-ac8690c4723c\") " pod="openshift-insights/insights-operator-5c847dcfc4-ngjjc"
Jul 07 11:31:26.562374 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.562277    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/331a680b-b6dd-46fd-b0da-cccfefd15bd3-service-ca-bundle\") pod \"router-default-5647bb9f4c-npcb5\" (UID: \"331a680b-b6dd-46fd-b0da-cccfefd15bd3\") " pod="openshift-ingress/router-default-5647bb9f4c-npcb5"
Jul 07 11:31:26.562374 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.562302    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zg468\" (UniqueName: \"kubernetes.io/projected/331a680b-b6dd-46fd-b0da-cccfefd15bd3-kube-api-access-zg468\") pod \"router-default-5647bb9f4c-npcb5\" (UID: \"331a680b-b6dd-46fd-b0da-cccfefd15bd3\") " pod="openshift-ingress/router-default-5647bb9f4c-npcb5"
Jul 07 11:31:26.562625 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.562382    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/faf6f343-6515-4b2c-a34e-286cd2d9ec7a-config\") pod \"console-operator-68b757fb67-ktbpw\" (UID: \"faf6f343-6515-4b2c-a34e-286cd2d9ec7a\") " pod="openshift-console-operator/console-operator-68b757fb67-ktbpw"
Jul 07 11:31:26.562625 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.562409    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/faf6f343-6515-4b2c-a34e-286cd2d9ec7a-trusted-ca\") pod \"console-operator-68b757fb67-ktbpw\" (UID: \"faf6f343-6515-4b2c-a34e-286cd2d9ec7a\") " pod="openshift-console-operator/console-operator-68b757fb67-ktbpw"
Jul 07 11:31:26.562625 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.562480    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxz9q\" (UniqueName: \"kubernetes.io/projected/faf6f343-6515-4b2c-a34e-286cd2d9ec7a-kube-api-access-dxz9q\") pod \"console-operator-68b757fb67-ktbpw\" (UID: \"faf6f343-6515-4b2c-a34e-286cd2d9ec7a\") " pod="openshift-console-operator/console-operator-68b757fb67-ktbpw"
Jul 07 11:31:26.562625 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.562510    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/331a680b-b6dd-46fd-b0da-cccfefd15bd3-default-certificate\") pod \"router-default-5647bb9f4c-npcb5\" (UID: \"331a680b-b6dd-46fd-b0da-cccfefd15bd3\") " pod="openshift-ingress/router-default-5647bb9f4c-npcb5"
Jul 07 11:31:26.562625 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.562537    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/faf6f343-6515-4b2c-a34e-286cd2d9ec7a-serving-cert\") pod \"console-operator-68b757fb67-ktbpw\" (UID: \"faf6f343-6515-4b2c-a34e-286cd2d9ec7a\") " pod="openshift-console-operator/console-operator-68b757fb67-ktbpw"
Jul 07 11:31:26.562625 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.562568    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/331a680b-b6dd-46fd-b0da-cccfefd15bd3-stats-auth\") pod \"router-default-5647bb9f4c-npcb5\" (UID: \"331a680b-b6dd-46fd-b0da-cccfefd15bd3\") " pod="openshift-ingress/router-default-5647bb9f4c-npcb5"
Jul 07 11:31:26.562915 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.562666    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49b59d5c-4db1-4013-8d2e-ac8690c4723c-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-ngjjc\" (UID: \"49b59d5c-4db1-4013-8d2e-ac8690c4723c\") " pod="openshift-insights/insights-operator-5c847dcfc4-ngjjc"
Jul 07 11:31:26.562915 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.562708    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/561f4302-647f-4dd2-bbcb-927a28c17df7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-ncgwv\" (UID: \"561f4302-647f-4dd2-bbcb-927a28c17df7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ncgwv"
Jul 07 11:31:26.562915 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.562736    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49b59d5c-4db1-4013-8d2e-ac8690c4723c-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-ngjjc\" (UID: \"49b59d5c-4db1-4013-8d2e-ac8690c4723c\") " pod="openshift-insights/insights-operator-5c847dcfc4-ngjjc"
Jul 07 11:31:26.562915 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.562764    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/49b59d5c-4db1-4013-8d2e-ac8690c4723c-snapshots\") pod \"insights-operator-5c847dcfc4-ngjjc\" (UID: \"49b59d5c-4db1-4013-8d2e-ac8690c4723c\") " pod="openshift-insights/insights-operator-5c847dcfc4-ngjjc"
Jul 07 11:31:26.562915 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.562794    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8vcx\" (UniqueName: \"kubernetes.io/projected/561f4302-647f-4dd2-bbcb-927a28c17df7-kube-api-access-q8vcx\") pod \"cluster-monitoring-operator-64786d9cbb-ncgwv\" (UID: \"561f4302-647f-4dd2-bbcb-927a28c17df7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ncgwv"
Jul 07 11:31:26.563071 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.563036    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/faf6f343-6515-4b2c-a34e-286cd2d9ec7a-config\") pod \"console-operator-68b757fb67-ktbpw\" (UID: \"faf6f343-6515-4b2c-a34e-286cd2d9ec7a\") " pod="openshift-console-operator/console-operator-68b757fb67-ktbpw"
Jul 07 11:31:26.563175 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.563156    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/faf6f343-6515-4b2c-a34e-286cd2d9ec7a-trusted-ca\") pod \"console-operator-68b757fb67-ktbpw\" (UID: \"faf6f343-6515-4b2c-a34e-286cd2d9ec7a\") " pod="openshift-console-operator/console-operator-68b757fb67-ktbpw"
Jul 07 11:31:26.564799 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.564782    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/faf6f343-6515-4b2c-a34e-286cd2d9ec7a-serving-cert\") pod \"console-operator-68b757fb67-ktbpw\" (UID: \"faf6f343-6515-4b2c-a34e-286cd2d9ec7a\") " pod="openshift-console-operator/console-operator-68b757fb67-ktbpw"
Jul 07 11:31:26.571682 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.571656    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxz9q\" (UniqueName: \"kubernetes.io/projected/faf6f343-6515-4b2c-a34e-286cd2d9ec7a-kube-api-access-dxz9q\") pod \"console-operator-68b757fb67-ktbpw\" (UID: \"faf6f343-6515-4b2c-a34e-286cd2d9ec7a\") " pod="openshift-console-operator/console-operator-68b757fb67-ktbpw"
Jul 07 11:31:26.663519 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.663435    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/331a680b-b6dd-46fd-b0da-cccfefd15bd3-service-ca-bundle\") pod \"router-default-5647bb9f4c-npcb5\" (UID: \"331a680b-b6dd-46fd-b0da-cccfefd15bd3\") " pod="openshift-ingress/router-default-5647bb9f4c-npcb5"
Jul 07 11:31:26.663519 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.663468    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zg468\" (UniqueName: \"kubernetes.io/projected/331a680b-b6dd-46fd-b0da-cccfefd15bd3-kube-api-access-zg468\") pod \"router-default-5647bb9f4c-npcb5\" (UID: \"331a680b-b6dd-46fd-b0da-cccfefd15bd3\") " pod="openshift-ingress/router-default-5647bb9f4c-npcb5"
Jul 07 11:31:26.663713 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:26.663618    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/331a680b-b6dd-46fd-b0da-cccfefd15bd3-service-ca-bundle podName:331a680b-b6dd-46fd-b0da-cccfefd15bd3 nodeName:}" failed. No retries permitted until 2026-07-07 11:31:27.163600446 +0000 UTC m=+275.137240833 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/331a680b-b6dd-46fd-b0da-cccfefd15bd3-service-ca-bundle") pod "router-default-5647bb9f4c-npcb5" (UID: "331a680b-b6dd-46fd-b0da-cccfefd15bd3") : configmap references non-existent config key: service-ca.crt
Jul 07 11:31:26.663713 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.663654    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/331a680b-b6dd-46fd-b0da-cccfefd15bd3-default-certificate\") pod \"router-default-5647bb9f4c-npcb5\" (UID: \"331a680b-b6dd-46fd-b0da-cccfefd15bd3\") " pod="openshift-ingress/router-default-5647bb9f4c-npcb5"
Jul 07 11:31:26.663713 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.663693    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/331a680b-b6dd-46fd-b0da-cccfefd15bd3-stats-auth\") pod \"router-default-5647bb9f4c-npcb5\" (UID: \"331a680b-b6dd-46fd-b0da-cccfefd15bd3\") " pod="openshift-ingress/router-default-5647bb9f4c-npcb5"
Jul 07 11:31:26.663713 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.663708    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49b59d5c-4db1-4013-8d2e-ac8690c4723c-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-ngjjc\" (UID: \"49b59d5c-4db1-4013-8d2e-ac8690c4723c\") " pod="openshift-insights/insights-operator-5c847dcfc4-ngjjc"
Jul 07 11:31:26.663910 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.663820    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/561f4302-647f-4dd2-bbcb-927a28c17df7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-ncgwv\" (UID: \"561f4302-647f-4dd2-bbcb-927a28c17df7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ncgwv"
Jul 07 11:31:26.663910 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.663855    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49b59d5c-4db1-4013-8d2e-ac8690c4723c-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-ngjjc\" (UID: \"49b59d5c-4db1-4013-8d2e-ac8690c4723c\") " pod="openshift-insights/insights-operator-5c847dcfc4-ngjjc"
Jul 07 11:31:26.663910 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.663884    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/49b59d5c-4db1-4013-8d2e-ac8690c4723c-snapshots\") pod \"insights-operator-5c847dcfc4-ngjjc\" (UID: \"49b59d5c-4db1-4013-8d2e-ac8690c4723c\") " pod="openshift-insights/insights-operator-5c847dcfc4-ngjjc"
Jul 07 11:31:26.663910 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:26.663889    2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 07 11:31:26.664120 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.663916    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8vcx\" (UniqueName: \"kubernetes.io/projected/561f4302-647f-4dd2-bbcb-927a28c17df7-kube-api-access-q8vcx\") pod \"cluster-monitoring-operator-64786d9cbb-ncgwv\" (UID: \"561f4302-647f-4dd2-bbcb-927a28c17df7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ncgwv"
Jul 07 11:31:26.664120 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:26.663951    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/561f4302-647f-4dd2-bbcb-927a28c17df7-cluster-monitoring-operator-tls podName:561f4302-647f-4dd2-bbcb-927a28c17df7 nodeName:}" failed. No retries permitted until 2026-07-07 11:31:27.163932831 +0000 UTC m=+275.137573229 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/561f4302-647f-4dd2-bbcb-927a28c17df7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-ncgwv" (UID: "561f4302-647f-4dd2-bbcb-927a28c17df7") : secret "cluster-monitoring-operator-tls" not found
Jul 07 11:31:26.664120 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.663978    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5742\" (UniqueName: \"kubernetes.io/projected/49b59d5c-4db1-4013-8d2e-ac8690c4723c-kube-api-access-x5742\") pod \"insights-operator-5c847dcfc4-ngjjc\" (UID: \"49b59d5c-4db1-4013-8d2e-ac8690c4723c\") " pod="openshift-insights/insights-operator-5c847dcfc4-ngjjc"
Jul 07 11:31:26.664120 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.664012    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/561f4302-647f-4dd2-bbcb-927a28c17df7-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-ncgwv\" (UID: \"561f4302-647f-4dd2-bbcb-927a28c17df7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ncgwv"
Jul 07 11:31:26.664120 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.664035    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/49b59d5c-4db1-4013-8d2e-ac8690c4723c-serving-cert\") pod \"insights-operator-5c847dcfc4-ngjjc\" (UID: \"49b59d5c-4db1-4013-8d2e-ac8690c4723c\") " pod="openshift-insights/insights-operator-5c847dcfc4-ngjjc"
Jul 07 11:31:26.664120 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.664081    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/331a680b-b6dd-46fd-b0da-cccfefd15bd3-metrics-certs\") pod \"router-default-5647bb9f4c-npcb5\" (UID: \"331a680b-b6dd-46fd-b0da-cccfefd15bd3\") " pod="openshift-ingress/router-default-5647bb9f4c-npcb5"
Jul 07 11:31:26.664120 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.664109    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/49b59d5c-4db1-4013-8d2e-ac8690c4723c-tmp\") pod \"insights-operator-5c847dcfc4-ngjjc\" (UID: \"49b59d5c-4db1-4013-8d2e-ac8690c4723c\") " pod="openshift-insights/insights-operator-5c847dcfc4-ngjjc"
Jul 07 11:31:26.664646 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.664472    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49b59d5c-4db1-4013-8d2e-ac8690c4723c-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-ngjjc\" (UID: \"49b59d5c-4db1-4013-8d2e-ac8690c4723c\") " pod="openshift-insights/insights-operator-5c847dcfc4-ngjjc"
Jul 07 11:31:26.664646 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:26.664558    2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 07 11:31:26.664646 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.664578    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/49b59d5c-4db1-4013-8d2e-ac8690c4723c-snapshots\") pod \"insights-operator-5c847dcfc4-ngjjc\" (UID: \"49b59d5c-4db1-4013-8d2e-ac8690c4723c\") " pod="openshift-insights/insights-operator-5c847dcfc4-ngjjc"
Jul 07 11:31:26.664646 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:26.664602    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/331a680b-b6dd-46fd-b0da-cccfefd15bd3-metrics-certs podName:331a680b-b6dd-46fd-b0da-cccfefd15bd3 nodeName:}" failed. No retries permitted until 2026-07-07 11:31:27.164587016 +0000 UTC m=+275.138227416 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/331a680b-b6dd-46fd-b0da-cccfefd15bd3-metrics-certs") pod "router-default-5647bb9f4c-npcb5" (UID: "331a680b-b6dd-46fd-b0da-cccfefd15bd3") : secret "router-metrics-certs-default" not found
Jul 07 11:31:26.664646 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.664586    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49b59d5c-4db1-4013-8d2e-ac8690c4723c-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-ngjjc\" (UID: \"49b59d5c-4db1-4013-8d2e-ac8690c4723c\") " pod="openshift-insights/insights-operator-5c847dcfc4-ngjjc"
Jul 07 11:31:26.664938 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.664843    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/49b59d5c-4db1-4013-8d2e-ac8690c4723c-tmp\") pod \"insights-operator-5c847dcfc4-ngjjc\" (UID: \"49b59d5c-4db1-4013-8d2e-ac8690c4723c\") " pod="openshift-insights/insights-operator-5c847dcfc4-ngjjc"
Jul 07 11:31:26.664938 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.664910    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/561f4302-647f-4dd2-bbcb-927a28c17df7-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-ncgwv\" (UID: \"561f4302-647f-4dd2-bbcb-927a28c17df7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ncgwv"
Jul 07 11:31:26.665867 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.665831    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-ktbpw"
Jul 07 11:31:26.666193 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.666173    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/331a680b-b6dd-46fd-b0da-cccfefd15bd3-stats-auth\") pod \"router-default-5647bb9f4c-npcb5\" (UID: \"331a680b-b6dd-46fd-b0da-cccfefd15bd3\") " pod="openshift-ingress/router-default-5647bb9f4c-npcb5"
Jul 07 11:31:26.666314 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.666294    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/331a680b-b6dd-46fd-b0da-cccfefd15bd3-default-certificate\") pod \"router-default-5647bb9f4c-npcb5\" (UID: \"331a680b-b6dd-46fd-b0da-cccfefd15bd3\") " pod="openshift-ingress/router-default-5647bb9f4c-npcb5"
Jul 07 11:31:26.666653 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.666632    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/49b59d5c-4db1-4013-8d2e-ac8690c4723c-serving-cert\") pod \"insights-operator-5c847dcfc4-ngjjc\" (UID: \"49b59d5c-4db1-4013-8d2e-ac8690c4723c\") " pod="openshift-insights/insights-operator-5c847dcfc4-ngjjc"
Jul 07 11:31:26.675948 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.675926    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zg468\" (UniqueName: \"kubernetes.io/projected/331a680b-b6dd-46fd-b0da-cccfefd15bd3-kube-api-access-zg468\") pod \"router-default-5647bb9f4c-npcb5\" (UID: \"331a680b-b6dd-46fd-b0da-cccfefd15bd3\") " pod="openshift-ingress/router-default-5647bb9f4c-npcb5"
Jul 07 11:31:26.676109 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.676088    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5742\" (UniqueName: \"kubernetes.io/projected/49b59d5c-4db1-4013-8d2e-ac8690c4723c-kube-api-access-x5742\") pod \"insights-operator-5c847dcfc4-ngjjc\" (UID: \"49b59d5c-4db1-4013-8d2e-ac8690c4723c\") " pod="openshift-insights/insights-operator-5c847dcfc4-ngjjc"
Jul 07 11:31:26.676305 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.676281    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8vcx\" (UniqueName: \"kubernetes.io/projected/561f4302-647f-4dd2-bbcb-927a28c17df7-kube-api-access-q8vcx\") pod \"cluster-monitoring-operator-64786d9cbb-ncgwv\" (UID: \"561f4302-647f-4dd2-bbcb-927a28c17df7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ncgwv"
Jul 07 11:31:26.773319 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.773285    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-ngjjc"
Jul 07 11:31:26.793693 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.793633    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-ktbpw"]
Jul 07 11:31:26.797449 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:31:26.797419    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfaf6f343_6515_4b2c_a34e_286cd2d9ec7a.slice/crio-98b603f5e0d6048b4eeb9ee80a669d033d56282730fb8828cd287beef334b7c1 WatchSource:0}: Error finding container 98b603f5e0d6048b4eeb9ee80a669d033d56282730fb8828cd287beef334b7c1: Status 404 returned error can't find the container with id 98b603f5e0d6048b4eeb9ee80a669d033d56282730fb8828cd287beef334b7c1
Jul 07 11:31:26.898754 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:26.898710    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-ngjjc"]
Jul 07 11:31:26.901591 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:31:26.901565    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49b59d5c_4db1_4013_8d2e_ac8690c4723c.slice/crio-6e3e7832557878f1cc78134b70f87a95c9d3ae9f60ba1be2ddbb95d029182247 WatchSource:0}: Error finding container 6e3e7832557878f1cc78134b70f87a95c9d3ae9f60ba1be2ddbb95d029182247: Status 404 returned error can't find the container with id 6e3e7832557878f1cc78134b70f87a95c9d3ae9f60ba1be2ddbb95d029182247
Jul 07 11:31:27.168312 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:27.168262    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/331a680b-b6dd-46fd-b0da-cccfefd15bd3-metrics-certs\") pod \"router-default-5647bb9f4c-npcb5\" (UID: \"331a680b-b6dd-46fd-b0da-cccfefd15bd3\") " pod="openshift-ingress/router-default-5647bb9f4c-npcb5"
Jul 07 11:31:27.168312 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:27.168311    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/331a680b-b6dd-46fd-b0da-cccfefd15bd3-service-ca-bundle\") pod \"router-default-5647bb9f4c-npcb5\" (UID: \"331a680b-b6dd-46fd-b0da-cccfefd15bd3\") " pod="openshift-ingress/router-default-5647bb9f4c-npcb5"
Jul 07 11:31:27.168530 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:27.168378    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/561f4302-647f-4dd2-bbcb-927a28c17df7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-ncgwv\" (UID: \"561f4302-647f-4dd2-bbcb-927a28c17df7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ncgwv"
Jul 07 11:31:27.168530 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:27.168415    2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 07 11:31:27.168530 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:27.168483    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/331a680b-b6dd-46fd-b0da-cccfefd15bd3-metrics-certs podName:331a680b-b6dd-46fd-b0da-cccfefd15bd3 nodeName:}" failed. No retries permitted until 2026-07-07 11:31:28.168467938 +0000 UTC m=+276.142108326 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/331a680b-b6dd-46fd-b0da-cccfefd15bd3-metrics-certs") pod "router-default-5647bb9f4c-npcb5" (UID: "331a680b-b6dd-46fd-b0da-cccfefd15bd3") : secret "router-metrics-certs-default" not found
Jul 07 11:31:27.168530 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:27.168481    2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 07 11:31:27.168530 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:27.168505    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/331a680b-b6dd-46fd-b0da-cccfefd15bd3-service-ca-bundle podName:331a680b-b6dd-46fd-b0da-cccfefd15bd3 nodeName:}" failed. No retries permitted until 2026-07-07 11:31:28.16848908 +0000 UTC m=+276.142129474 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/331a680b-b6dd-46fd-b0da-cccfefd15bd3-service-ca-bundle") pod "router-default-5647bb9f4c-npcb5" (UID: "331a680b-b6dd-46fd-b0da-cccfefd15bd3") : configmap references non-existent config key: service-ca.crt
Jul 07 11:31:27.168530 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:27.168532    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/561f4302-647f-4dd2-bbcb-927a28c17df7-cluster-monitoring-operator-tls podName:561f4302-647f-4dd2-bbcb-927a28c17df7 nodeName:}" failed. No retries permitted until 2026-07-07 11:31:28.168521121 +0000 UTC m=+276.142161522 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/561f4302-647f-4dd2-bbcb-927a28c17df7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-ncgwv" (UID: "561f4302-647f-4dd2-bbcb-927a28c17df7") : secret "cluster-monitoring-operator-tls" not found
Jul 07 11:31:27.732422 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:27.732335    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-ktbpw" event={"ID":"faf6f343-6515-4b2c-a34e-286cd2d9ec7a","Type":"ContainerStarted","Data":"98b603f5e0d6048b4eeb9ee80a669d033d56282730fb8828cd287beef334b7c1"}
Jul 07 11:31:27.733785 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:27.733757    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-ngjjc" event={"ID":"49b59d5c-4db1-4013-8d2e-ac8690c4723c","Type":"ContainerStarted","Data":"6e3e7832557878f1cc78134b70f87a95c9d3ae9f60ba1be2ddbb95d029182247"}
Jul 07 11:31:28.178121 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:28.178032    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/331a680b-b6dd-46fd-b0da-cccfefd15bd3-service-ca-bundle\") pod \"router-default-5647bb9f4c-npcb5\" (UID: \"331a680b-b6dd-46fd-b0da-cccfefd15bd3\") " pod="openshift-ingress/router-default-5647bb9f4c-npcb5"
Jul 07 11:31:28.178325 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:28.178126    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/561f4302-647f-4dd2-bbcb-927a28c17df7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-ncgwv\" (UID: \"561f4302-647f-4dd2-bbcb-927a28c17df7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ncgwv"
Jul 07 11:31:28.178325 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:28.178184    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/331a680b-b6dd-46fd-b0da-cccfefd15bd3-metrics-certs\") pod \"router-default-5647bb9f4c-npcb5\" (UID: \"331a680b-b6dd-46fd-b0da-cccfefd15bd3\") " pod="openshift-ingress/router-default-5647bb9f4c-npcb5"
Jul 07 11:31:28.178325 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:28.178234    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/331a680b-b6dd-46fd-b0da-cccfefd15bd3-service-ca-bundle podName:331a680b-b6dd-46fd-b0da-cccfefd15bd3 nodeName:}" failed. No retries permitted until 2026-07-07 11:31:30.178196613 +0000 UTC m=+278.151837002 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/331a680b-b6dd-46fd-b0da-cccfefd15bd3-service-ca-bundle") pod "router-default-5647bb9f4c-npcb5" (UID: "331a680b-b6dd-46fd-b0da-cccfefd15bd3") : configmap references non-existent config key: service-ca.crt
Jul 07 11:31:28.178325 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:28.178289    2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 07 11:31:28.178552 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:28.178340    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/561f4302-647f-4dd2-bbcb-927a28c17df7-cluster-monitoring-operator-tls podName:561f4302-647f-4dd2-bbcb-927a28c17df7 nodeName:}" failed. No retries permitted until 2026-07-07 11:31:30.178325085 +0000 UTC m=+278.151965492 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/561f4302-647f-4dd2-bbcb-927a28c17df7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-ncgwv" (UID: "561f4302-647f-4dd2-bbcb-927a28c17df7") : secret "cluster-monitoring-operator-tls" not found
Jul 07 11:31:28.178552 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:28.178289    2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 07 11:31:28.178552 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:28.178390    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/331a680b-b6dd-46fd-b0da-cccfefd15bd3-metrics-certs podName:331a680b-b6dd-46fd-b0da-cccfefd15bd3 nodeName:}" failed. No retries permitted until 2026-07-07 11:31:30.178380216 +0000 UTC m=+278.152020610 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/331a680b-b6dd-46fd-b0da-cccfefd15bd3-metrics-certs") pod "router-default-5647bb9f4c-npcb5" (UID: "331a680b-b6dd-46fd-b0da-cccfefd15bd3") : secret "router-metrics-certs-default" not found
Jul 07 11:31:29.742359 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:29.742330    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ktbpw_faf6f343-6515-4b2c-a34e-286cd2d9ec7a/console-operator/0.log"
Jul 07 11:31:29.742810 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:29.742367    2570 generic.go:356] "Generic (PLEG): container finished" podID="faf6f343-6515-4b2c-a34e-286cd2d9ec7a" containerID="b30935c5d8b576f2f7e1ff0c8bb5e980877ae3756781c7a388ab3c99671eaafd" exitCode=255
Jul 07 11:31:29.742810 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:29.742401    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-ktbpw" event={"ID":"faf6f343-6515-4b2c-a34e-286cd2d9ec7a","Type":"ContainerDied","Data":"b30935c5d8b576f2f7e1ff0c8bb5e980877ae3756781c7a388ab3c99671eaafd"}
Jul 07 11:31:29.742810 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:29.742670    2570 scope.go:117] "RemoveContainer" containerID="b30935c5d8b576f2f7e1ff0c8bb5e980877ae3756781c7a388ab3c99671eaafd"
Jul 07 11:31:29.744024 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:29.743999    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-ngjjc" event={"ID":"49b59d5c-4db1-4013-8d2e-ac8690c4723c","Type":"ContainerStarted","Data":"0de11e630a5e85f9835184cccbdbcc99f55fd402d346b8f7d5cfef4cea8df141"}
Jul 07 11:31:30.039764 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:30.039657    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-ngjjc" podStartSLOduration=1.664220508 podStartE2EDuration="4.039631714s" totalImagesPullingTime="2.375411206s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:31:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:31:29.774555896 +0000 UTC m=+277.748196307" watchObservedRunningTime="2026-07-07 11:31:30.039631714 +0000 UTC m=+278.013272126"
Jul 07 11:31:30.040653 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:30.040631    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-r4q9n"]
Jul 07 11:31:30.043373 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:30.043346    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-r4q9n"
Jul 07 11:31:30.048531 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:30.048504    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bv7rg\""
Jul 07 11:31:30.048651 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:30.048531    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 07 11:31:30.050024 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:30.049929    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 07 11:31:30.065880 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:30.065846    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-r4q9n"]
Jul 07 11:31:30.197676 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:30.197630    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r4q9n\" (UID: \"1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32\") " pod="openshift-insights/insights-runtime-extractor-r4q9n"
Jul 07 11:31:30.197676 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:30.197672    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4sj92\" (UniqueName: \"kubernetes.io/projected/1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32-kube-api-access-4sj92\") pod \"insights-runtime-extractor-r4q9n\" (UID: \"1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32\") " pod="openshift-insights/insights-runtime-extractor-r4q9n"
Jul 07 11:31:30.197899 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:30.197691    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32-data-volume\") pod \"insights-runtime-extractor-r4q9n\" (UID: \"1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32\") " pod="openshift-insights/insights-runtime-extractor-r4q9n"
Jul 07 11:31:30.197899 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:30.197723    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/561f4302-647f-4dd2-bbcb-927a28c17df7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-ncgwv\" (UID: \"561f4302-647f-4dd2-bbcb-927a28c17df7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ncgwv"
Jul 07 11:31:30.197899 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:30.197756    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r4q9n\" (UID: \"1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32\") " pod="openshift-insights/insights-runtime-extractor-r4q9n"
Jul 07 11:31:30.197899 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:30.197791    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32-crio-socket\") pod \"insights-runtime-extractor-r4q9n\" (UID: \"1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32\") " pod="openshift-insights/insights-runtime-extractor-r4q9n"
Jul 07 11:31:30.197899 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:30.197820    2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 07 11:31:30.197899 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:30.197819    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/331a680b-b6dd-46fd-b0da-cccfefd15bd3-metrics-certs\") pod \"router-default-5647bb9f4c-npcb5\" (UID: \"331a680b-b6dd-46fd-b0da-cccfefd15bd3\") " pod="openshift-ingress/router-default-5647bb9f4c-npcb5"
Jul 07 11:31:30.197899 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:30.197886    2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 07 11:31:30.198103 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:30.197912    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/561f4302-647f-4dd2-bbcb-927a28c17df7-cluster-monitoring-operator-tls podName:561f4302-647f-4dd2-bbcb-927a28c17df7 nodeName:}" failed. No retries permitted until 2026-07-07 11:31:34.197887857 +0000 UTC m=+282.171528248 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/561f4302-647f-4dd2-bbcb-927a28c17df7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-ncgwv" (UID: "561f4302-647f-4dd2-bbcb-927a28c17df7") : secret "cluster-monitoring-operator-tls" not found
Jul 07 11:31:30.198103 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:30.197955    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/331a680b-b6dd-46fd-b0da-cccfefd15bd3-metrics-certs podName:331a680b-b6dd-46fd-b0da-cccfefd15bd3 nodeName:}" failed. No retries permitted until 2026-07-07 11:31:34.197942148 +0000 UTC m=+282.171582540 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/331a680b-b6dd-46fd-b0da-cccfefd15bd3-metrics-certs") pod "router-default-5647bb9f4c-npcb5" (UID: "331a680b-b6dd-46fd-b0da-cccfefd15bd3") : secret "router-metrics-certs-default" not found
Jul 07 11:31:30.198103 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:30.198002    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/331a680b-b6dd-46fd-b0da-cccfefd15bd3-service-ca-bundle\") pod \"router-default-5647bb9f4c-npcb5\" (UID: \"331a680b-b6dd-46fd-b0da-cccfefd15bd3\") " pod="openshift-ingress/router-default-5647bb9f4c-npcb5"
Jul 07 11:31:30.198103 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:30.198083    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/331a680b-b6dd-46fd-b0da-cccfefd15bd3-service-ca-bundle podName:331a680b-b6dd-46fd-b0da-cccfefd15bd3 nodeName:}" failed. No retries permitted until 2026-07-07 11:31:34.198075409 +0000 UTC m=+282.171715800 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/331a680b-b6dd-46fd-b0da-cccfefd15bd3-service-ca-bundle") pod "router-default-5647bb9f4c-npcb5" (UID: "331a680b-b6dd-46fd-b0da-cccfefd15bd3") : configmap references non-existent config key: service-ca.crt
Jul 07 11:31:30.298806 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:30.298717    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32-crio-socket\") pod \"insights-runtime-extractor-r4q9n\" (UID: \"1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32\") " pod="openshift-insights/insights-runtime-extractor-r4q9n"
Jul 07 11:31:30.298964 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:30.298833    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r4q9n\" (UID: \"1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32\") " pod="openshift-insights/insights-runtime-extractor-r4q9n"
Jul 07 11:31:30.298964 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:30.298857    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4sj92\" (UniqueName: \"kubernetes.io/projected/1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32-kube-api-access-4sj92\") pod \"insights-runtime-extractor-r4q9n\" (UID: \"1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32\") " pod="openshift-insights/insights-runtime-extractor-r4q9n"
Jul 07 11:31:30.298964 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:30.298858    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32-crio-socket\") pod \"insights-runtime-extractor-r4q9n\" (UID: \"1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32\") " pod="openshift-insights/insights-runtime-extractor-r4q9n"
Jul 07 11:31:30.298964 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:30.298876    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32-data-volume\") pod \"insights-runtime-extractor-r4q9n\" (UID: \"1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32\") " pod="openshift-insights/insights-runtime-extractor-r4q9n"
Jul 07 11:31:30.299234 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:30.298963    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r4q9n\" (UID: \"1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32\") " pod="openshift-insights/insights-runtime-extractor-r4q9n"
Jul 07 11:31:30.299234 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:30.298984    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 11:31:30.299234 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:30.299044    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32-insights-runtime-extractor-tls podName:1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32 nodeName:}" failed. No retries permitted until 2026-07-07 11:31:30.799025965 +0000 UTC m=+278.772666375 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r4q9n" (UID: "1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32") : secret "insights-runtime-extractor-tls" not found
Jul 07 11:31:30.299234 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:30.299087    2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 11:31:30.299234 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:30.299150    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32-data-volume\") pod \"insights-runtime-extractor-r4q9n\" (UID: \"1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32\") " pod="openshift-insights/insights-runtime-extractor-r4q9n"
Jul 07 11:31:30.299234 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:30.299165    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32-kube-rbac-proxy-cm podName:1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32 nodeName:}" failed. No retries permitted until 2026-07-07 11:31:30.799145978 +0000 UTC m=+278.772786373 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32-kube-rbac-proxy-cm") pod "insights-runtime-extractor-r4q9n" (UID: "1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32") : configmap "kube-rbac-proxy" not found
Jul 07 11:31:30.307719 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:30.307694    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4sj92\" (UniqueName: \"kubernetes.io/projected/1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32-kube-api-access-4sj92\") pod \"insights-runtime-extractor-r4q9n\" (UID: \"1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32\") " pod="openshift-insights/insights-runtime-extractor-r4q9n"
Jul 07 11:31:30.748845 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:30.748814    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ktbpw_faf6f343-6515-4b2c-a34e-286cd2d9ec7a/console-operator/1.log"
Jul 07 11:31:30.749302 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:30.749238    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ktbpw_faf6f343-6515-4b2c-a34e-286cd2d9ec7a/console-operator/0.log"
Jul 07 11:31:30.749302 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:30.749277    2570 generic.go:356] "Generic (PLEG): container finished" podID="faf6f343-6515-4b2c-a34e-286cd2d9ec7a" containerID="0167506aef855a1efe39726d3e1df36c18cb0ca3cf4be48a72779040fcd01194" exitCode=255
Jul 07 11:31:30.749410 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:30.749357    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-ktbpw" event={"ID":"faf6f343-6515-4b2c-a34e-286cd2d9ec7a","Type":"ContainerDied","Data":"0167506aef855a1efe39726d3e1df36c18cb0ca3cf4be48a72779040fcd01194"}
Jul 07 11:31:30.749460 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:30.749414    2570 scope.go:117] "RemoveContainer" containerID="b30935c5d8b576f2f7e1ff0c8bb5e980877ae3756781c7a388ab3c99671eaafd"
Jul 07 11:31:30.749643 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:30.749622    2570 scope.go:117] "RemoveContainer" containerID="0167506aef855a1efe39726d3e1df36c18cb0ca3cf4be48a72779040fcd01194"
Jul 07 11:31:30.749839 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:30.749822    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-ktbpw_openshift-console-operator(faf6f343-6515-4b2c-a34e-286cd2d9ec7a)\"" pod="openshift-console-operator/console-operator-68b757fb67-ktbpw" podUID="faf6f343-6515-4b2c-a34e-286cd2d9ec7a"
Jul 07 11:31:30.803473 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:30.803431    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r4q9n\" (UID: \"1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32\") " pod="openshift-insights/insights-runtime-extractor-r4q9n"
Jul 07 11:31:30.803667 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:30.803507    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r4q9n\" (UID: \"1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32\") " pod="openshift-insights/insights-runtime-extractor-r4q9n"
Jul 07 11:31:30.803667 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:30.803596    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 11:31:30.803667 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:30.803630    2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 11:31:30.803667 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:30.803663    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32-insights-runtime-extractor-tls podName:1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32 nodeName:}" failed. No retries permitted until 2026-07-07 11:31:31.803647357 +0000 UTC m=+279.777287748 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r4q9n" (UID: "1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32") : secret "insights-runtime-extractor-tls" not found
Jul 07 11:31:30.803821 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:30.803677    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32-kube-rbac-proxy-cm podName:1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32 nodeName:}" failed. No retries permitted until 2026-07-07 11:31:31.803670862 +0000 UTC m=+279.777311250 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32-kube-rbac-proxy-cm") pod "insights-runtime-extractor-r4q9n" (UID: "1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32") : configmap "kube-rbac-proxy" not found
Jul 07 11:31:31.754124 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:31.754097    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ktbpw_faf6f343-6515-4b2c-a34e-286cd2d9ec7a/console-operator/1.log"
Jul 07 11:31:31.754520 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:31.754438    2570 scope.go:117] "RemoveContainer" containerID="0167506aef855a1efe39726d3e1df36c18cb0ca3cf4be48a72779040fcd01194"
Jul 07 11:31:31.754602 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:31.754580    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-ktbpw_openshift-console-operator(faf6f343-6515-4b2c-a34e-286cd2d9ec7a)\"" pod="openshift-console-operator/console-operator-68b757fb67-ktbpw" podUID="faf6f343-6515-4b2c-a34e-286cd2d9ec7a"
Jul 07 11:31:31.813516 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:31.813478    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r4q9n\" (UID: \"1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32\") " pod="openshift-insights/insights-runtime-extractor-r4q9n"
Jul 07 11:31:31.813687 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:31.813548    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r4q9n\" (UID: \"1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32\") " pod="openshift-insights/insights-runtime-extractor-r4q9n"
Jul 07 11:31:31.813687 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:31.813626    2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 11:31:31.813760 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:31.813694    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32-kube-rbac-proxy-cm podName:1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32 nodeName:}" failed. No retries permitted until 2026-07-07 11:31:33.813678073 +0000 UTC m=+281.787318477 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32-kube-rbac-proxy-cm") pod "insights-runtime-extractor-r4q9n" (UID: "1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32") : configmap "kube-rbac-proxy" not found
Jul 07 11:31:31.813760 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:31.813627    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 11:31:31.813828 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:31.813774    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32-insights-runtime-extractor-tls podName:1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32 nodeName:}" failed. No retries permitted until 2026-07-07 11:31:33.813757463 +0000 UTC m=+281.787397855 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r4q9n" (UID: "1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32") : secret "insights-runtime-extractor-tls" not found
Jul 07 11:31:32.228753 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:32.228725    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rkn2t_650c2a15-194e-47f4-99c8-9f94802d1e9b/dns-node-resolver/0.log"
Jul 07 11:31:33.028804 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:33.028778    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qh2dm_e54ef86d-1806-4e84-a235-aba912f88f58/node-ca/0.log"
Jul 07 11:31:33.294976 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:33.294899    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-h27wv"]
Jul 07 11:31:33.297389 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:33.297373    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-h27wv"
Jul 07 11:31:33.300130 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:33.300103    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 07 11:31:33.300277 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:33.300196    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-6v9bc\""
Jul 07 11:31:33.300277 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:33.300196    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 07 11:31:33.301504 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:33.301488    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 07 11:31:33.301504 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:33.301497    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 07 11:31:33.306332 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:33.306307    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-h27wv"]
Jul 07 11:31:33.427645 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:33.427602    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcs55\" (UniqueName: \"kubernetes.io/projected/66eb880a-449a-4b7f-96a3-c18eec6d9c46-kube-api-access-qcs55\") pod \"service-ca-758f599c5-h27wv\" (UID: \"66eb880a-449a-4b7f-96a3-c18eec6d9c46\") " pod="openshift-service-ca/service-ca-758f599c5-h27wv"
Jul 07 11:31:33.427819 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:33.427654    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/66eb880a-449a-4b7f-96a3-c18eec6d9c46-signing-cabundle\") pod \"service-ca-758f599c5-h27wv\" (UID: \"66eb880a-449a-4b7f-96a3-c18eec6d9c46\") " pod="openshift-service-ca/service-ca-758f599c5-h27wv"
Jul 07 11:31:33.427819 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:33.427786    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/66eb880a-449a-4b7f-96a3-c18eec6d9c46-signing-key\") pod \"service-ca-758f599c5-h27wv\" (UID: \"66eb880a-449a-4b7f-96a3-c18eec6d9c46\") " pod="openshift-service-ca/service-ca-758f599c5-h27wv"
Jul 07 11:31:33.528264 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:33.528199    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/66eb880a-449a-4b7f-96a3-c18eec6d9c46-signing-key\") pod \"service-ca-758f599c5-h27wv\" (UID: \"66eb880a-449a-4b7f-96a3-c18eec6d9c46\") " pod="openshift-service-ca/service-ca-758f599c5-h27wv"
Jul 07 11:31:33.528458 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:33.528335    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcs55\" (UniqueName: \"kubernetes.io/projected/66eb880a-449a-4b7f-96a3-c18eec6d9c46-kube-api-access-qcs55\") pod \"service-ca-758f599c5-h27wv\" (UID: \"66eb880a-449a-4b7f-96a3-c18eec6d9c46\") " pod="openshift-service-ca/service-ca-758f599c5-h27wv"
Jul 07 11:31:33.528458 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:33.528373    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/66eb880a-449a-4b7f-96a3-c18eec6d9c46-signing-cabundle\") pod \"service-ca-758f599c5-h27wv\" (UID: \"66eb880a-449a-4b7f-96a3-c18eec6d9c46\") " pod="openshift-service-ca/service-ca-758f599c5-h27wv"
Jul 07 11:31:33.528992 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:33.528969    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/66eb880a-449a-4b7f-96a3-c18eec6d9c46-signing-cabundle\") pod \"service-ca-758f599c5-h27wv\" (UID: \"66eb880a-449a-4b7f-96a3-c18eec6d9c46\") " pod="openshift-service-ca/service-ca-758f599c5-h27wv"
Jul 07 11:31:33.530558 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:33.530537    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/66eb880a-449a-4b7f-96a3-c18eec6d9c46-signing-key\") pod \"service-ca-758f599c5-h27wv\" (UID: \"66eb880a-449a-4b7f-96a3-c18eec6d9c46\") " pod="openshift-service-ca/service-ca-758f599c5-h27wv"
Jul 07 11:31:33.536631 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:33.536602    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcs55\" (UniqueName: \"kubernetes.io/projected/66eb880a-449a-4b7f-96a3-c18eec6d9c46-kube-api-access-qcs55\") pod \"service-ca-758f599c5-h27wv\" (UID: \"66eb880a-449a-4b7f-96a3-c18eec6d9c46\") " pod="openshift-service-ca/service-ca-758f599c5-h27wv"
Jul 07 11:31:33.608188 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:33.608101    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-h27wv"
Jul 07 11:31:33.732369 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:33.732339    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-h27wv"]
Jul 07 11:31:33.735104 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:31:33.735079    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66eb880a_449a_4b7f_96a3_c18eec6d9c46.slice/crio-b4c9980f97718359d389f6b5f5378338e60c99494c31ae0c5bc9a4e0aa6e3477 WatchSource:0}: Error finding container b4c9980f97718359d389f6b5f5378338e60c99494c31ae0c5bc9a4e0aa6e3477: Status 404 returned error can't find the container with id b4c9980f97718359d389f6b5f5378338e60c99494c31ae0c5bc9a4e0aa6e3477
Jul 07 11:31:33.761314 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:33.761273    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-h27wv" event={"ID":"66eb880a-449a-4b7f-96a3-c18eec6d9c46","Type":"ContainerStarted","Data":"b4c9980f97718359d389f6b5f5378338e60c99494c31ae0c5bc9a4e0aa6e3477"}
Jul 07 11:31:33.831228 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:33.831187    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r4q9n\" (UID: \"1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32\") " pod="openshift-insights/insights-runtime-extractor-r4q9n"
Jul 07 11:31:33.831388 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:33.831305    2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 11:31:33.831388 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:33.831364    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r4q9n\" (UID: \"1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32\") " pod="openshift-insights/insights-runtime-extractor-r4q9n"
Jul 07 11:31:33.831388 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:33.831373    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32-kube-rbac-proxy-cm podName:1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32 nodeName:}" failed. No retries permitted until 2026-07-07 11:31:37.83135534 +0000 UTC m=+285.804995751 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32-kube-rbac-proxy-cm") pod "insights-runtime-extractor-r4q9n" (UID: "1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32") : configmap "kube-rbac-proxy" not found
Jul 07 11:31:33.831526 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:33.831507    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 11:31:33.831579 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:33.831567    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32-insights-runtime-extractor-tls podName:1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32 nodeName:}" failed. No retries permitted until 2026-07-07 11:31:37.831553186 +0000 UTC m=+285.805193574 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r4q9n" (UID: "1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32") : secret "insights-runtime-extractor-tls" not found
Jul 07 11:31:34.029273 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:34.029244    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zd288_24d1f3b4-0970-41a7-95e6-6ab8e00f5b47/serve-healthcheck-canary/0.log"
Jul 07 11:31:34.234364 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:34.234326    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/561f4302-647f-4dd2-bbcb-927a28c17df7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-ncgwv\" (UID: \"561f4302-647f-4dd2-bbcb-927a28c17df7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ncgwv"
Jul 07 11:31:34.234537 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:34.234399    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/331a680b-b6dd-46fd-b0da-cccfefd15bd3-metrics-certs\") pod \"router-default-5647bb9f4c-npcb5\" (UID: \"331a680b-b6dd-46fd-b0da-cccfefd15bd3\") " pod="openshift-ingress/router-default-5647bb9f4c-npcb5"
Jul 07 11:31:34.234537 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:34.234421    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/331a680b-b6dd-46fd-b0da-cccfefd15bd3-service-ca-bundle\") pod \"router-default-5647bb9f4c-npcb5\" (UID: \"331a680b-b6dd-46fd-b0da-cccfefd15bd3\") " pod="openshift-ingress/router-default-5647bb9f4c-npcb5"
Jul 07 11:31:34.234537 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:34.234463    2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 07 11:31:34.234537 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:34.234530    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/561f4302-647f-4dd2-bbcb-927a28c17df7-cluster-monitoring-operator-tls podName:561f4302-647f-4dd2-bbcb-927a28c17df7 nodeName:}" failed. No retries permitted until 2026-07-07 11:31:42.234514327 +0000 UTC m=+290.208154715 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/561f4302-647f-4dd2-bbcb-927a28c17df7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-ncgwv" (UID: "561f4302-647f-4dd2-bbcb-927a28c17df7") : secret "cluster-monitoring-operator-tls" not found
Jul 07 11:31:34.234675 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:34.234538    2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 07 11:31:34.234675 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:34.234546    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/331a680b-b6dd-46fd-b0da-cccfefd15bd3-service-ca-bundle podName:331a680b-b6dd-46fd-b0da-cccfefd15bd3 nodeName:}" failed. No retries permitted until 2026-07-07 11:31:42.234535542 +0000 UTC m=+290.208175930 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/331a680b-b6dd-46fd-b0da-cccfefd15bd3-service-ca-bundle") pod "router-default-5647bb9f4c-npcb5" (UID: "331a680b-b6dd-46fd-b0da-cccfefd15bd3") : configmap references non-existent config key: service-ca.crt
Jul 07 11:31:34.234675 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:34.234584    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/331a680b-b6dd-46fd-b0da-cccfefd15bd3-metrics-certs podName:331a680b-b6dd-46fd-b0da-cccfefd15bd3 nodeName:}" failed. No retries permitted until 2026-07-07 11:31:42.234569564 +0000 UTC m=+290.208209953 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/331a680b-b6dd-46fd-b0da-cccfefd15bd3-metrics-certs") pod "router-default-5647bb9f4c-npcb5" (UID: "331a680b-b6dd-46fd-b0da-cccfefd15bd3") : secret "router-metrics-certs-default" not found
Jul 07 11:31:35.457510 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:35.457469    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-696d746fbb-8plv4" podUID="1aa5a3e9-5a49-4837-8fd2-389682ccef44"
Jul 07 11:31:35.457510 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:35.457470    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6bffdf9778-wlk99" podUID="88a181af-4d7c-4398-a53b-69ab8f2e4035"
Jul 07 11:31:35.457994 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:35.457469    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-9gqzt" podUID="bfcc5fcf-866f-405b-ae62-dcf401627bc7"
Jul 07 11:31:35.766907 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:35.766879    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-696d746fbb-8plv4"
Jul 07 11:31:35.767036 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:35.766879    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9gqzt"
Jul 07 11:31:35.767099 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:35.766879    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-wlk99"
Jul 07 11:31:36.666677 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:36.666583    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-ktbpw"
Jul 07 11:31:36.666677 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:36.666647    2570 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-ktbpw"
Jul 07 11:31:36.667078 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:36.666973    2570 scope.go:117] "RemoveContainer" containerID="0167506aef855a1efe39726d3e1df36c18cb0ca3cf4be48a72779040fcd01194"
Jul 07 11:31:36.667139 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:36.667115    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-ktbpw_openshift-console-operator(faf6f343-6515-4b2c-a34e-286cd2d9ec7a)\"" pod="openshift-console-operator/console-operator-68b757fb67-ktbpw" podUID="faf6f343-6515-4b2c-a34e-286cd2d9ec7a"
Jul 07 11:31:36.772709 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:36.772674    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-h27wv" event={"ID":"66eb880a-449a-4b7f-96a3-c18eec6d9c46","Type":"ContainerStarted","Data":"d626c48977776d76b22168a4160096a22d75ed40ffcca32f189923dd67876133"}
Jul 07 11:31:36.793192 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:36.793142    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-h27wv" podStartSLOduration=1.7723447879999998 podStartE2EDuration="3.793124664s" totalImagesPullingTime="2.020779876s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:31:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:31:36.792449785 +0000 UTC m=+284.766090206" watchObservedRunningTime="2026-07-07 11:31:36.793124664 +0000 UTC m=+284.766765073"
Jul 07 11:31:37.867274 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:37.867229    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r4q9n\" (UID: \"1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32\") " pod="openshift-insights/insights-runtime-extractor-r4q9n"
Jul 07 11:31:37.867768 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:37.867354    2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 11:31:37.867768 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:37.867450    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32-kube-rbac-proxy-cm podName:1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32 nodeName:}" failed. No retries permitted until 2026-07-07 11:31:45.867427923 +0000 UTC m=+293.841068314 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32-kube-rbac-proxy-cm") pod "insights-runtime-extractor-r4q9n" (UID: "1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32") : configmap "kube-rbac-proxy" not found
Jul 07 11:31:37.867768 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:37.867596    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r4q9n\" (UID: \"1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32\") " pod="openshift-insights/insights-runtime-extractor-r4q9n"
Jul 07 11:31:37.867978 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:37.867960    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 11:31:37.868191 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:37.868175    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32-insights-runtime-extractor-tls podName:1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32 nodeName:}" failed. No retries permitted until 2026-07-07 11:31:45.868157376 +0000 UTC m=+293.841797768 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r4q9n" (UID: "1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32") : secret "insights-runtime-extractor-tls" not found
Jul 07 11:31:39.279683 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:39.279624    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/88a181af-4d7c-4398-a53b-69ab8f2e4035-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-wlk99\" (UID: \"88a181af-4d7c-4398-a53b-69ab8f2e4035\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-wlk99"
Jul 07 11:31:39.280119 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:39.279706    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1aa5a3e9-5a49-4837-8fd2-389682ccef44-registry-tls\") pod \"image-registry-696d746fbb-8plv4\" (UID: \"1aa5a3e9-5a49-4837-8fd2-389682ccef44\") " pod="openshift-image-registry/image-registry-696d746fbb-8plv4"
Jul 07 11:31:39.280119 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:39.279738    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bfcc5fcf-866f-405b-ae62-dcf401627bc7-metrics-tls\") pod \"dns-default-9gqzt\" (UID: \"bfcc5fcf-866f-405b-ae62-dcf401627bc7\") " pod="openshift-dns/dns-default-9gqzt"
Jul 07 11:31:39.280119 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:39.279794    2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 07 11:31:39.280119 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:39.279827    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 11:31:39.280119 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:39.279838    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 07 11:31:39.280119 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:39.279853    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-696d746fbb-8plv4: secret "image-registry-tls" not found
Jul 07 11:31:39.280119 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:39.279878    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88a181af-4d7c-4398-a53b-69ab8f2e4035-networking-console-plugin-cert podName:88a181af-4d7c-4398-a53b-69ab8f2e4035 nodeName:}" failed. No retries permitted until 2026-07-07 11:33:41.279859274 +0000 UTC m=+409.253499685 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/88a181af-4d7c-4398-a53b-69ab8f2e4035-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-wlk99" (UID: "88a181af-4d7c-4398-a53b-69ab8f2e4035") : secret "networking-console-plugin-cert" not found
Jul 07 11:31:39.280119 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:39.279896    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1aa5a3e9-5a49-4837-8fd2-389682ccef44-registry-tls podName:1aa5a3e9-5a49-4837-8fd2-389682ccef44 nodeName:}" failed. No retries permitted until 2026-07-07 11:33:41.279886985 +0000 UTC m=+409.253527375 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1aa5a3e9-5a49-4837-8fd2-389682ccef44-registry-tls") pod "image-registry-696d746fbb-8plv4" (UID: "1aa5a3e9-5a49-4837-8fd2-389682ccef44") : secret "image-registry-tls" not found
Jul 07 11:31:39.280119 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:39.279907    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfcc5fcf-866f-405b-ae62-dcf401627bc7-metrics-tls podName:bfcc5fcf-866f-405b-ae62-dcf401627bc7 nodeName:}" failed. No retries permitted until 2026-07-07 11:33:41.279901592 +0000 UTC m=+409.253541980 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bfcc5fcf-866f-405b-ae62-dcf401627bc7-metrics-tls") pod "dns-default-9gqzt" (UID: "bfcc5fcf-866f-405b-ae62-dcf401627bc7") : secret "dns-default-metrics-tls" not found
Jul 07 11:31:42.305055 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:42.305006    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/561f4302-647f-4dd2-bbcb-927a28c17df7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-ncgwv\" (UID: \"561f4302-647f-4dd2-bbcb-927a28c17df7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ncgwv"
Jul 07 11:31:42.305575 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:42.305091    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/331a680b-b6dd-46fd-b0da-cccfefd15bd3-metrics-certs\") pod \"router-default-5647bb9f4c-npcb5\" (UID: \"331a680b-b6dd-46fd-b0da-cccfefd15bd3\") " pod="openshift-ingress/router-default-5647bb9f4c-npcb5"
Jul 07 11:31:42.305575 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:42.305120    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/331a680b-b6dd-46fd-b0da-cccfefd15bd3-service-ca-bundle\") pod \"router-default-5647bb9f4c-npcb5\" (UID: \"331a680b-b6dd-46fd-b0da-cccfefd15bd3\") " pod="openshift-ingress/router-default-5647bb9f4c-npcb5"
Jul 07 11:31:42.305575 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:42.305149    2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 07 11:31:42.305575 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:42.305229    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/561f4302-647f-4dd2-bbcb-927a28c17df7-cluster-monitoring-operator-tls podName:561f4302-647f-4dd2-bbcb-927a28c17df7 nodeName:}" failed. No retries permitted until 2026-07-07 11:31:58.305193578 +0000 UTC m=+306.278833966 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/561f4302-647f-4dd2-bbcb-927a28c17df7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-ncgwv" (UID: "561f4302-647f-4dd2-bbcb-927a28c17df7") : secret "cluster-monitoring-operator-tls" not found
Jul 07 11:31:42.305575 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:42.305249    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/331a680b-b6dd-46fd-b0da-cccfefd15bd3-service-ca-bundle podName:331a680b-b6dd-46fd-b0da-cccfefd15bd3 nodeName:}" failed. No retries permitted until 2026-07-07 11:31:58.305237198 +0000 UTC m=+306.278877586 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/331a680b-b6dd-46fd-b0da-cccfefd15bd3-service-ca-bundle") pod "router-default-5647bb9f4c-npcb5" (UID: "331a680b-b6dd-46fd-b0da-cccfefd15bd3") : configmap references non-existent config key: service-ca.crt
Jul 07 11:31:42.307484 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:42.307461    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/331a680b-b6dd-46fd-b0da-cccfefd15bd3-metrics-certs\") pod \"router-default-5647bb9f4c-npcb5\" (UID: \"331a680b-b6dd-46fd-b0da-cccfefd15bd3\") " pod="openshift-ingress/router-default-5647bb9f4c-npcb5"
Jul 07 11:31:45.939394 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:45.939355    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r4q9n\" (UID: \"1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32\") " pod="openshift-insights/insights-runtime-extractor-r4q9n"
Jul 07 11:31:45.939764 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:45.939411    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r4q9n\" (UID: \"1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32\") " pod="openshift-insights/insights-runtime-extractor-r4q9n"
Jul 07 11:31:45.939764 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:45.939493    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 11:31:45.939764 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:45.939564    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32-insights-runtime-extractor-tls podName:1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32 nodeName:}" failed. No retries permitted until 2026-07-07 11:32:01.93954851 +0000 UTC m=+309.913188898 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r4q9n" (UID: "1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32") : secret "insights-runtime-extractor-tls" not found
Jul 07 11:31:45.939931 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:45.939869    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r4q9n\" (UID: \"1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32\") " pod="openshift-insights/insights-runtime-extractor-r4q9n"
Jul 07 11:31:49.933424 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:31:49.933382    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-ppmbz" podUID="f75e11d0-c0de-4a8d-849d-807a87aa1497"
Jul 07 11:31:50.932109 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:50.932072    2570 scope.go:117] "RemoveContainer" containerID="0167506aef855a1efe39726d3e1df36c18cb0ca3cf4be48a72779040fcd01194"
Jul 07 11:31:51.815949 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:51.815919    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ktbpw_faf6f343-6515-4b2c-a34e-286cd2d9ec7a/console-operator/1.log"
Jul 07 11:31:51.816369 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:51.815981    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-ktbpw" event={"ID":"faf6f343-6515-4b2c-a34e-286cd2d9ec7a","Type":"ContainerStarted","Data":"23a65d6393d65fb0fd5d428ac1e77627611af1fde8d4d8f6ec288fa4f50bf5d2"}
Jul 07 11:31:51.816369 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:51.816260    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-ktbpw"
Jul 07 11:31:51.835499 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:51.835448    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-ktbpw" podStartSLOduration=23.357078136 podStartE2EDuration="25.835432708s" totalImagesPullingTime="2.478354572s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:31:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:31:51.834693717 +0000 UTC m=+299.808334128" watchObservedRunningTime="2026-07-07 11:31:51.835432708 +0000 UTC m=+299.809073117"
Jul 07 11:31:51.956853 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:51.956825    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-ktbpw"
Jul 07 11:31:53.826619 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:53.826585    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ktbpw_faf6f343-6515-4b2c-a34e-286cd2d9ec7a/console-operator/1.log"
Jul 07 11:31:53.829580 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:53.829552    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ktbpw_faf6f343-6515-4b2c-a34e-286cd2d9ec7a/console-operator/1.log"
Jul 07 11:31:53.837023 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:53.837003    2570 kubelet.go:1648] "Image garbage collection succeeded"
Jul 07 11:31:58.342519 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:58.342456    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/331a680b-b6dd-46fd-b0da-cccfefd15bd3-service-ca-bundle\") pod \"router-default-5647bb9f4c-npcb5\" (UID: \"331a680b-b6dd-46fd-b0da-cccfefd15bd3\") " pod="openshift-ingress/router-default-5647bb9f4c-npcb5"
Jul 07 11:31:58.344027 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:58.342571    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/561f4302-647f-4dd2-bbcb-927a28c17df7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-ncgwv\" (UID: \"561f4302-647f-4dd2-bbcb-927a28c17df7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ncgwv"
Jul 07 11:31:58.344027 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:58.343062    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/331a680b-b6dd-46fd-b0da-cccfefd15bd3-service-ca-bundle\") pod \"router-default-5647bb9f4c-npcb5\" (UID: \"331a680b-b6dd-46fd-b0da-cccfefd15bd3\") " pod="openshift-ingress/router-default-5647bb9f4c-npcb5"
Jul 07 11:31:58.344990 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:58.344969    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/561f4302-647f-4dd2-bbcb-927a28c17df7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-ncgwv\" (UID: \"561f4302-647f-4dd2-bbcb-927a28c17df7\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ncgwv"
Jul 07 11:31:58.566267 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:58.566236    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-mdkrb\""
Jul 07 11:31:58.573760 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:58.573736    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ncgwv"
Jul 07 11:31:58.584391 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:58.584358    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-zshjf\""
Jul 07 11:31:58.592224 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:58.592184    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5647bb9f4c-npcb5"
Jul 07 11:31:58.711573 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:58.711402    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ncgwv"]
Jul 07 11:31:58.714004 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:31:58.713975    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod561f4302_647f_4dd2_bbcb_927a28c17df7.slice/crio-b38d216ee94b307945f9ff5fbb69b988720832227dd3b8896658d23836f9155c WatchSource:0}: Error finding container b38d216ee94b307945f9ff5fbb69b988720832227dd3b8896658d23836f9155c: Status 404 returned error can't find the container with id b38d216ee94b307945f9ff5fbb69b988720832227dd3b8896658d23836f9155c
Jul 07 11:31:58.716013 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:58.715998    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 11:31:58.727091 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:58.726982    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5647bb9f4c-npcb5"]
Jul 07 11:31:58.729360 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:31:58.729339    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod331a680b_b6dd_46fd_b0da_cccfefd15bd3.slice/crio-6e462a3471323980a599fc1cd8c83e90172cb32c9fe87806370b0cef5c740f08 WatchSource:0}: Error finding container 6e462a3471323980a599fc1cd8c83e90172cb32c9fe87806370b0cef5c740f08: Status 404 returned error can't find the container with id 6e462a3471323980a599fc1cd8c83e90172cb32c9fe87806370b0cef5c740f08
Jul 07 11:31:58.837614 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:58.837577    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5647bb9f4c-npcb5" event={"ID":"331a680b-b6dd-46fd-b0da-cccfefd15bd3","Type":"ContainerStarted","Data":"afb82e7e0323c08071f76c6724a6e92424d5a2ad44b4564186183abde799d361"}
Jul 07 11:31:58.837799 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:58.837628    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5647bb9f4c-npcb5" event={"ID":"331a680b-b6dd-46fd-b0da-cccfefd15bd3","Type":"ContainerStarted","Data":"6e462a3471323980a599fc1cd8c83e90172cb32c9fe87806370b0cef5c740f08"}
Jul 07 11:31:58.838700 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:58.838676    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ncgwv" event={"ID":"561f4302-647f-4dd2-bbcb-927a28c17df7","Type":"ContainerStarted","Data":"b38d216ee94b307945f9ff5fbb69b988720832227dd3b8896658d23836f9155c"}
Jul 07 11:31:58.856506 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:58.856417    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-5647bb9f4c-npcb5" podStartSLOduration=32.856398871 podStartE2EDuration="32.856398871s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:31:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:31:58.85577167 +0000 UTC m=+306.829412079" watchObservedRunningTime="2026-07-07 11:31:58.856398871 +0000 UTC m=+306.830039260"
Jul 07 11:31:59.592566 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:59.592527    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-5647bb9f4c-npcb5"
Jul 07 11:31:59.596346 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:59.596322    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-5647bb9f4c-npcb5"
Jul 07 11:31:59.842354 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:59.842323    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-5647bb9f4c-npcb5"
Jul 07 11:31:59.843822 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:31:59.843753    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-5647bb9f4c-npcb5"
Jul 07 11:32:00.847777 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:00.847690    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ncgwv" event={"ID":"561f4302-647f-4dd2-bbcb-927a28c17df7","Type":"ContainerStarted","Data":"16e505a4c2bbe0c6a75ffa0f864174396af4c7a85f7e4d9a83ef6076c802d505"}
Jul 07 11:32:00.866152 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:00.866091    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ncgwv" podStartSLOduration=33.021258493 podStartE2EDuration="34.866072669s" totalImagesPullingTime="1.844814176s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:31:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:32:00.865497976 +0000 UTC m=+308.839138409" watchObservedRunningTime="2026-07-07 11:32:00.866072669 +0000 UTC m=+308.839713079"
Jul 07 11:32:01.037548 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:01.037515    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-c6mxv"]
Jul 07 11:32:01.039582 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:01.039559    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-c6mxv"
Jul 07 11:32:01.042774 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:01.042749    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 07 11:32:01.042922 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:01.042911    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-4bmfm\""
Jul 07 11:32:01.048950 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:01.048923    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-c6mxv"]
Jul 07 11:32:01.167943 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:01.167845    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ce03fa2b-19e6-407c-ad1b-474032191b71-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-c6mxv\" (UID: \"ce03fa2b-19e6-407c-ad1b-474032191b71\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-c6mxv"
Jul 07 11:32:01.268347 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:01.268312    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ce03fa2b-19e6-407c-ad1b-474032191b71-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-c6mxv\" (UID: \"ce03fa2b-19e6-407c-ad1b-474032191b71\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-c6mxv"
Jul 07 11:32:01.270720 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:01.270693    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ce03fa2b-19e6-407c-ad1b-474032191b71-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-c6mxv\" (UID: \"ce03fa2b-19e6-407c-ad1b-474032191b71\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-c6mxv"
Jul 07 11:32:01.348882 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:01.348842    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-c6mxv"
Jul 07 11:32:01.472149 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:01.471974    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-c6mxv"]
Jul 07 11:32:01.479713 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:32:01.479584    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce03fa2b_19e6_407c_ad1b_474032191b71.slice/crio-036c2639838d3600feb15b3b1bb4fc53ed53420a5c378bf4994e7e24cd12fc68 WatchSource:0}: Error finding container 036c2639838d3600feb15b3b1bb4fc53ed53420a5c378bf4994e7e24cd12fc68: Status 404 returned error can't find the container with id 036c2639838d3600feb15b3b1bb4fc53ed53420a5c378bf4994e7e24cd12fc68
Jul 07 11:32:01.851972 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:01.851936    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-c6mxv" event={"ID":"ce03fa2b-19e6-407c-ad1b-474032191b71","Type":"ContainerStarted","Data":"036c2639838d3600feb15b3b1bb4fc53ed53420a5c378bf4994e7e24cd12fc68"}
Jul 07 11:32:01.973553 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:01.973513    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r4q9n\" (UID: \"1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32\") " pod="openshift-insights/insights-runtime-extractor-r4q9n"
Jul 07 11:32:01.976083 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:01.976058    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r4q9n\" (UID: \"1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32\") " pod="openshift-insights/insights-runtime-extractor-r4q9n"
Jul 07 11:32:02.155688 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:02.155611    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bv7rg\""
Jul 07 11:32:02.163133 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:02.163097    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-r4q9n"
Jul 07 11:32:02.301479 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:02.301451    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-r4q9n"]
Jul 07 11:32:02.459540 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:32:02.459506    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b5d28c2_dcc8_4ad9_a0f4_5f36a5d67e32.slice/crio-a02824b2fafeee5afe73a185ccd14bc0556dfd38a7014a7118437f48c41de9ca WatchSource:0}: Error finding container a02824b2fafeee5afe73a185ccd14bc0556dfd38a7014a7118437f48c41de9ca: Status 404 returned error can't find the container with id a02824b2fafeee5afe73a185ccd14bc0556dfd38a7014a7118437f48c41de9ca
Jul 07 11:32:02.856146 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:02.856049    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r4q9n" event={"ID":"1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32","Type":"ContainerStarted","Data":"a02824b2fafeee5afe73a185ccd14bc0556dfd38a7014a7118437f48c41de9ca"}
Jul 07 11:32:02.857388 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:02.857367    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-c6mxv" event={"ID":"ce03fa2b-19e6-407c-ad1b-474032191b71","Type":"ContainerStarted","Data":"10d53011e30636be854108f558700a8de28aec386143a7be328340bb3c6cee24"}
Jul 07 11:32:02.857629 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:02.857613    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-c6mxv"
Jul 07 11:32:02.862305 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:02.862283    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-c6mxv"
Jul 07 11:32:02.875609 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:02.875557    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-c6mxv" podStartSLOduration=0.874559355 podStartE2EDuration="1.875539082s" totalImagesPullingTime="1.000979727s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:32:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:32:02.875039654 +0000 UTC m=+310.848680065" watchObservedRunningTime="2026-07-07 11:32:02.875539082 +0000 UTC m=+310.849179493"
Jul 07 11:32:02.933339 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:02.933067    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ppmbz"
Jul 07 11:32:03.109489 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:03.108068    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-kf2j7"]
Jul 07 11:32:03.110786 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:03.110589    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kf2j7"
Jul 07 11:32:03.114454 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:03.114140    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 07 11:32:03.124684 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:03.123797    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 07 11:32:03.124684 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:03.123913    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-65cqr\""
Jul 07 11:32:03.124684 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:03.124199    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 07 11:32:03.138073 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:03.136522    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-kf2j7"]
Jul 07 11:32:03.185668 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:03.185623    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1fcc05da-8891-4f1f-bbb7-4e886316c910-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-kf2j7\" (UID: \"1fcc05da-8891-4f1f-bbb7-4e886316c910\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kf2j7"
Jul 07 11:32:03.185863 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:03.185704    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mx8dr\" (UniqueName: \"kubernetes.io/projected/1fcc05da-8891-4f1f-bbb7-4e886316c910-kube-api-access-mx8dr\") pod \"prometheus-operator-567f7b8b7c-kf2j7\" (UID: \"1fcc05da-8891-4f1f-bbb7-4e886316c910\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kf2j7"
Jul 07 11:32:03.185928 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:03.185873    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1fcc05da-8891-4f1f-bbb7-4e886316c910-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-kf2j7\" (UID: \"1fcc05da-8891-4f1f-bbb7-4e886316c910\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kf2j7"
Jul 07 11:32:03.185928 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:03.185909    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1fcc05da-8891-4f1f-bbb7-4e886316c910-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-kf2j7\" (UID: \"1fcc05da-8891-4f1f-bbb7-4e886316c910\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kf2j7"
Jul 07 11:32:03.287928 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:03.287366    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1fcc05da-8891-4f1f-bbb7-4e886316c910-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-kf2j7\" (UID: \"1fcc05da-8891-4f1f-bbb7-4e886316c910\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kf2j7"
Jul 07 11:32:03.287928 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:03.287413    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1fcc05da-8891-4f1f-bbb7-4e886316c910-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-kf2j7\" (UID: \"1fcc05da-8891-4f1f-bbb7-4e886316c910\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kf2j7"
Jul 07 11:32:03.287928 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:03.287459    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1fcc05da-8891-4f1f-bbb7-4e886316c910-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-kf2j7\" (UID: \"1fcc05da-8891-4f1f-bbb7-4e886316c910\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kf2j7"
Jul 07 11:32:03.287928 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:03.287510    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mx8dr\" (UniqueName: \"kubernetes.io/projected/1fcc05da-8891-4f1f-bbb7-4e886316c910-kube-api-access-mx8dr\") pod \"prometheus-operator-567f7b8b7c-kf2j7\" (UID: \"1fcc05da-8891-4f1f-bbb7-4e886316c910\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kf2j7"
Jul 07 11:32:03.288336 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:32:03.287954    2570 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 07 11:32:03.288336 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:32:03.288014    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1fcc05da-8891-4f1f-bbb7-4e886316c910-prometheus-operator-tls podName:1fcc05da-8891-4f1f-bbb7-4e886316c910 nodeName:}" failed. No retries permitted until 2026-07-07 11:32:03.787995998 +0000 UTC m=+311.761636387 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/1fcc05da-8891-4f1f-bbb7-4e886316c910-prometheus-operator-tls") pod "prometheus-operator-567f7b8b7c-kf2j7" (UID: "1fcc05da-8891-4f1f-bbb7-4e886316c910") : secret "prometheus-operator-tls" not found
Jul 07 11:32:03.288739 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:03.288707    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1fcc05da-8891-4f1f-bbb7-4e886316c910-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-kf2j7\" (UID: \"1fcc05da-8891-4f1f-bbb7-4e886316c910\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kf2j7"
Jul 07 11:32:03.291197 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:03.291168    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1fcc05da-8891-4f1f-bbb7-4e886316c910-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-kf2j7\" (UID: \"1fcc05da-8891-4f1f-bbb7-4e886316c910\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kf2j7"
Jul 07 11:32:03.310036 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:03.309983    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mx8dr\" (UniqueName: \"kubernetes.io/projected/1fcc05da-8891-4f1f-bbb7-4e886316c910-kube-api-access-mx8dr\") pod \"prometheus-operator-567f7b8b7c-kf2j7\" (UID: \"1fcc05da-8891-4f1f-bbb7-4e886316c910\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kf2j7"
Jul 07 11:32:03.791865 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:03.791824    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1fcc05da-8891-4f1f-bbb7-4e886316c910-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-kf2j7\" (UID: \"1fcc05da-8891-4f1f-bbb7-4e886316c910\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kf2j7"
Jul 07 11:32:03.792074 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:32:03.791978    2570 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 07 11:32:03.792074 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:32:03.792059    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1fcc05da-8891-4f1f-bbb7-4e886316c910-prometheus-operator-tls podName:1fcc05da-8891-4f1f-bbb7-4e886316c910 nodeName:}" failed. No retries permitted until 2026-07-07 11:32:04.792037963 +0000 UTC m=+312.765678351 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/1fcc05da-8891-4f1f-bbb7-4e886316c910-prometheus-operator-tls") pod "prometheus-operator-567f7b8b7c-kf2j7" (UID: "1fcc05da-8891-4f1f-bbb7-4e886316c910") : secret "prometheus-operator-tls" not found
Jul 07 11:32:04.799391 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:04.799355    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1fcc05da-8891-4f1f-bbb7-4e886316c910-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-kf2j7\" (UID: \"1fcc05da-8891-4f1f-bbb7-4e886316c910\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kf2j7"
Jul 07 11:32:04.801963 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:04.801933    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1fcc05da-8891-4f1f-bbb7-4e886316c910-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-kf2j7\" (UID: \"1fcc05da-8891-4f1f-bbb7-4e886316c910\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kf2j7"
Jul 07 11:32:04.923632 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:04.923587    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kf2j7"
Jul 07 11:32:06.273877 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:06.273851    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-kf2j7"]
Jul 07 11:32:06.277396 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:32:06.277366    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fcc05da_8891_4f1f_bbb7_4e886316c910.slice/crio-8032c00638a77777f44869e5324ff9105fbbacb469cd7f8d74970146f7e4089d WatchSource:0}: Error finding container 8032c00638a77777f44869e5324ff9105fbbacb469cd7f8d74970146f7e4089d: Status 404 returned error can't find the container with id 8032c00638a77777f44869e5324ff9105fbbacb469cd7f8d74970146f7e4089d
Jul 07 11:32:06.872068 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:06.872035    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r4q9n" event={"ID":"1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32","Type":"ContainerStarted","Data":"ee29097fe244399284a69619b17222a9f5cf6146ff5c98cbb66073824309dd51"}
Jul 07 11:32:06.872190 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:06.872095    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r4q9n" event={"ID":"1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32","Type":"ContainerStarted","Data":"d1a95ee6574d9ece5be91374fda51962fe12a89a8e88e265d2e37ddc6afecdb8"}
Jul 07 11:32:06.873633 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:06.873606    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kf2j7" event={"ID":"1fcc05da-8891-4f1f-bbb7-4e886316c910","Type":"ContainerStarted","Data":"8032c00638a77777f44869e5324ff9105fbbacb469cd7f8d74970146f7e4089d"}
Jul 07 11:32:07.881296 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:07.881198    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kf2j7" event={"ID":"1fcc05da-8891-4f1f-bbb7-4e886316c910","Type":"ContainerStarted","Data":"d703d1782794e55a4f1519a7137da2d56383b51a2b5b1b9412998e193ed2b4b8"}
Jul 07 11:32:08.887567 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:08.887525    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kf2j7" event={"ID":"1fcc05da-8891-4f1f-bbb7-4e886316c910","Type":"ContainerStarted","Data":"9b697b783789cfcf3e9a56ca869c4aafcbfbcb417de64050d1852050b66de513"}
Jul 07 11:32:08.905912 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:08.905848    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-kf2j7" podStartSLOduration=4.439352445 podStartE2EDuration="5.905824665s" totalImagesPullingTime="1.46647222s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:32:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:32:08.90530935 +0000 UTC m=+316.878949761" watchObservedRunningTime="2026-07-07 11:32:08.905824665 +0000 UTC m=+316.879465081"
Jul 07 11:32:10.524688 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.524648    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-gwvbz"]
Jul 07 11:32:10.528491 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.528470    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-gwvbz"
Jul 07 11:32:10.554827 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.554796    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 07 11:32:10.555062 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.555037    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 07 11:32:10.555378 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.555346    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-5f899\""
Jul 07 11:32:10.640607 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.640568    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j4488"]
Jul 07 11:32:10.643749 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.643732    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j4488"
Jul 07 11:32:10.647609 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.647580    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 07 11:32:10.647868 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.647849    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-w42qq\""
Jul 07 11:32:10.648143 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.648124    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 07 11:32:10.648903 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.648886    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 07 11:32:10.652850 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.652828    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blv2l\" (UniqueName: \"kubernetes.io/projected/3e9e44ad-ca0e-4ae8-af94-0d51e0259ce2-kube-api-access-blv2l\") pod \"openshift-state-metrics-54cf96dfcf-gwvbz\" (UID: \"3e9e44ad-ca0e-4ae8-af94-0d51e0259ce2\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-gwvbz"
Jul 07 11:32:10.652944 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.652869    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3e9e44ad-ca0e-4ae8-af94-0d51e0259ce2-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-gwvbz\" (UID: \"3e9e44ad-ca0e-4ae8-af94-0d51e0259ce2\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-gwvbz"
Jul 07 11:32:10.652944 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.652914    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3e9e44ad-ca0e-4ae8-af94-0d51e0259ce2-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-gwvbz\" (UID: \"3e9e44ad-ca0e-4ae8-af94-0d51e0259ce2\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-gwvbz"
Jul 07 11:32:10.653057 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.653037    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e9e44ad-ca0e-4ae8-af94-0d51e0259ce2-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-gwvbz\" (UID: \"3e9e44ad-ca0e-4ae8-af94-0d51e0259ce2\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-gwvbz"
Jul 07 11:32:10.658935 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.656242    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-gwvbz"]
Jul 07 11:32:10.662062 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.662032    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-b9rvb"]
Jul 07 11:32:10.665514 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.665490    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-b9rvb"
Jul 07 11:32:10.668687 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.668507    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 07 11:32:10.668687 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.668566    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 07 11:32:10.669016 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.668998    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-9pw78\""
Jul 07 11:32:10.669933 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.669914    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j4488"]
Jul 07 11:32:10.671954 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.671936    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 07 11:32:10.754004 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.753964    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/54a5035f-302e-4f61-a39a-cf4eebf2f669-root\") pod \"node-exporter-b9rvb\" (UID: \"54a5035f-302e-4f61-a39a-cf4eebf2f669\") " pod="openshift-monitoring/node-exporter-b9rvb"
Jul 07 11:32:10.754004 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.754000    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/54a5035f-302e-4f61-a39a-cf4eebf2f669-node-exporter-tls\") pod \"node-exporter-b9rvb\" (UID: \"54a5035f-302e-4f61-a39a-cf4eebf2f669\") " pod="openshift-monitoring/node-exporter-b9rvb"
Jul 07 11:32:10.754233 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.754066    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/54a5035f-302e-4f61-a39a-cf4eebf2f669-node-exporter-textfile\") pod \"node-exporter-b9rvb\" (UID: \"54a5035f-302e-4f61-a39a-cf4eebf2f669\") " pod="openshift-monitoring/node-exporter-b9rvb"
Jul 07 11:32:10.754233 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.754094    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e9e44ad-ca0e-4ae8-af94-0d51e0259ce2-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-gwvbz\" (UID: \"3e9e44ad-ca0e-4ae8-af94-0d51e0259ce2\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-gwvbz"
Jul 07 11:32:10.754233 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.754149    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blv2l\" (UniqueName: \"kubernetes.io/projected/3e9e44ad-ca0e-4ae8-af94-0d51e0259ce2-kube-api-access-blv2l\") pod \"openshift-state-metrics-54cf96dfcf-gwvbz\" (UID: \"3e9e44ad-ca0e-4ae8-af94-0d51e0259ce2\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-gwvbz"
Jul 07 11:32:10.754233 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.754189    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3e9e44ad-ca0e-4ae8-af94-0d51e0259ce2-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-gwvbz\" (UID: \"3e9e44ad-ca0e-4ae8-af94-0d51e0259ce2\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-gwvbz"
Jul 07 11:32:10.754433 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.754236    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/54a5035f-302e-4f61-a39a-cf4eebf2f669-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b9rvb\" (UID: \"54a5035f-302e-4f61-a39a-cf4eebf2f669\") " pod="openshift-monitoring/node-exporter-b9rvb"
Jul 07 11:32:10.754433 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.754264    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/54a5035f-302e-4f61-a39a-cf4eebf2f669-node-exporter-accelerators-collector-config\") pod \"node-exporter-b9rvb\" (UID: \"54a5035f-302e-4f61-a39a-cf4eebf2f669\") " pod="openshift-monitoring/node-exporter-b9rvb"
Jul 07 11:32:10.754433 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.754316    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/54a5035f-302e-4f61-a39a-cf4eebf2f669-node-exporter-wtmp\") pod \"node-exporter-b9rvb\" (UID: \"54a5035f-302e-4f61-a39a-cf4eebf2f669\") " pod="openshift-monitoring/node-exporter-b9rvb"
Jul 07 11:32:10.754433 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.754351    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/54a5035f-302e-4f61-a39a-cf4eebf2f669-sys\") pod \"node-exporter-b9rvb\" (UID: \"54a5035f-302e-4f61-a39a-cf4eebf2f669\") " pod="openshift-monitoring/node-exporter-b9rvb"
Jul 07 11:32:10.754433 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.754375    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfmjn\" (UniqueName: \"kubernetes.io/projected/f8e91d6e-7866-4576-bae6-a351d600d703-kube-api-access-wfmjn\") pod \"kube-state-metrics-7c9dbb8b7d-j4488\" (UID: \"f8e91d6e-7866-4576-bae6-a351d600d703\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j4488"
Jul 07 11:32:10.754433 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.754418    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3e9e44ad-ca0e-4ae8-af94-0d51e0259ce2-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-gwvbz\" (UID: \"3e9e44ad-ca0e-4ae8-af94-0d51e0259ce2\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-gwvbz"
Jul 07 11:32:10.754433 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.754439    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f8e91d6e-7866-4576-bae6-a351d600d703-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-j4488\" (UID: \"f8e91d6e-7866-4576-bae6-a351d600d703\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j4488"
Jul 07 11:32:10.754433 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.754466    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/f8e91d6e-7866-4576-bae6-a351d600d703-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-j4488\" (UID: \"f8e91d6e-7866-4576-bae6-a351d600d703\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j4488"
Jul 07 11:32:10.754891 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.754497    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f8e91d6e-7866-4576-bae6-a351d600d703-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-j4488\" (UID: \"f8e91d6e-7866-4576-bae6-a351d600d703\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j4488"
Jul 07 11:32:10.754891 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.754528    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/54a5035f-302e-4f61-a39a-cf4eebf2f669-metrics-client-ca\") pod \"node-exporter-b9rvb\" (UID: \"54a5035f-302e-4f61-a39a-cf4eebf2f669\") " pod="openshift-monitoring/node-exporter-b9rvb"
Jul 07 11:32:10.754891 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.754556    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74d5b\" (UniqueName: \"kubernetes.io/projected/54a5035f-302e-4f61-a39a-cf4eebf2f669-kube-api-access-74d5b\") pod \"node-exporter-b9rvb\" (UID: \"54a5035f-302e-4f61-a39a-cf4eebf2f669\") " pod="openshift-monitoring/node-exporter-b9rvb"
Jul 07 11:32:10.754891 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.754579    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/f8e91d6e-7866-4576-bae6-a351d600d703-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-j4488\" (UID: \"f8e91d6e-7866-4576-bae6-a351d600d703\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j4488"
Jul 07 11:32:10.754891 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.754661    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f8e91d6e-7866-4576-bae6-a351d600d703-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-j4488\" (UID: \"f8e91d6e-7866-4576-bae6-a351d600d703\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j4488"
Jul 07 11:32:10.755064 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.754924    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e9e44ad-ca0e-4ae8-af94-0d51e0259ce2-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-gwvbz\" (UID: \"3e9e44ad-ca0e-4ae8-af94-0d51e0259ce2\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-gwvbz"
Jul 07 11:32:10.756877 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.756842    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3e9e44ad-ca0e-4ae8-af94-0d51e0259ce2-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-gwvbz\" (UID: \"3e9e44ad-ca0e-4ae8-af94-0d51e0259ce2\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-gwvbz"
Jul 07 11:32:10.756980 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.756919    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3e9e44ad-ca0e-4ae8-af94-0d51e0259ce2-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-gwvbz\" (UID: \"3e9e44ad-ca0e-4ae8-af94-0d51e0259ce2\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-gwvbz"
Jul 07 11:32:10.765141 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.765116    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-blv2l\" (UniqueName: \"kubernetes.io/projected/3e9e44ad-ca0e-4ae8-af94-0d51e0259ce2-kube-api-access-blv2l\") pod \"openshift-state-metrics-54cf96dfcf-gwvbz\" (UID: \"3e9e44ad-ca0e-4ae8-af94-0d51e0259ce2\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-gwvbz"
Jul 07 11:32:10.838136 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.838034    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-gwvbz"
Jul 07 11:32:10.855530 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.855499    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f8e91d6e-7866-4576-bae6-a351d600d703-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-j4488\" (UID: \"f8e91d6e-7866-4576-bae6-a351d600d703\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j4488"
Jul 07 11:32:10.855688 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.855536    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/54a5035f-302e-4f61-a39a-cf4eebf2f669-metrics-client-ca\") pod \"node-exporter-b9rvb\" (UID: \"54a5035f-302e-4f61-a39a-cf4eebf2f669\") " pod="openshift-monitoring/node-exporter-b9rvb"
Jul 07 11:32:10.855688 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.855555    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74d5b\" (UniqueName: \"kubernetes.io/projected/54a5035f-302e-4f61-a39a-cf4eebf2f669-kube-api-access-74d5b\") pod \"node-exporter-b9rvb\" (UID: \"54a5035f-302e-4f61-a39a-cf4eebf2f669\") " pod="openshift-monitoring/node-exporter-b9rvb"
Jul 07 11:32:10.855688 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.855579    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/f8e91d6e-7866-4576-bae6-a351d600d703-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-j4488\" (UID: \"f8e91d6e-7866-4576-bae6-a351d600d703\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j4488"
Jul 07 11:32:10.855688 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.855613    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f8e91d6e-7866-4576-bae6-a351d600d703-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-j4488\" (UID: \"f8e91d6e-7866-4576-bae6-a351d600d703\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j4488"
Jul 07 11:32:10.855688 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.855655    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/54a5035f-302e-4f61-a39a-cf4eebf2f669-root\") pod \"node-exporter-b9rvb\" (UID: \"54a5035f-302e-4f61-a39a-cf4eebf2f669\") " pod="openshift-monitoring/node-exporter-b9rvb"
Jul 07 11:32:10.855957 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.855679    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/54a5035f-302e-4f61-a39a-cf4eebf2f669-node-exporter-tls\") pod \"node-exporter-b9rvb\" (UID: \"54a5035f-302e-4f61-a39a-cf4eebf2f669\") " pod="openshift-monitoring/node-exporter-b9rvb"
Jul 07 11:32:10.856014 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.855968    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/54a5035f-302e-4f61-a39a-cf4eebf2f669-root\") pod \"node-exporter-b9rvb\" (UID: \"54a5035f-302e-4f61-a39a-cf4eebf2f669\") " pod="openshift-monitoring/node-exporter-b9rvb"
Jul 07 11:32:10.856093 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.856070    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/f8e91d6e-7866-4576-bae6-a351d600d703-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-j4488\" (UID: \"f8e91d6e-7866-4576-bae6-a351d600d703\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j4488"
Jul 07 11:32:10.856757 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.856470    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f8e91d6e-7866-4576-bae6-a351d600d703-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-j4488\" (UID: \"f8e91d6e-7866-4576-bae6-a351d600d703\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j4488"
Jul 07 11:32:10.856757 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.856547    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/54a5035f-302e-4f61-a39a-cf4eebf2f669-node-exporter-textfile\") pod \"node-exporter-b9rvb\" (UID: \"54a5035f-302e-4f61-a39a-cf4eebf2f669\") " pod="openshift-monitoring/node-exporter-b9rvb"
Jul 07 11:32:10.856757 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.856614    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/54a5035f-302e-4f61-a39a-cf4eebf2f669-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b9rvb\" (UID: \"54a5035f-302e-4f61-a39a-cf4eebf2f669\") " pod="openshift-monitoring/node-exporter-b9rvb"
Jul 07 11:32:10.856757 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.856668    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/54a5035f-302e-4f61-a39a-cf4eebf2f669-node-exporter-accelerators-collector-config\") pod \"node-exporter-b9rvb\" (UID: \"54a5035f-302e-4f61-a39a-cf4eebf2f669\") " pod="openshift-monitoring/node-exporter-b9rvb"
Jul 07 11:32:10.856757 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.856706    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/54a5035f-302e-4f61-a39a-cf4eebf2f669-node-exporter-wtmp\") pod \"node-exporter-b9rvb\" (UID: \"54a5035f-302e-4f61-a39a-cf4eebf2f669\") " pod="openshift-monitoring/node-exporter-b9rvb"
Jul 07 11:32:10.856757 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.856739    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/54a5035f-302e-4f61-a39a-cf4eebf2f669-sys\") pod \"node-exporter-b9rvb\" (UID: \"54a5035f-302e-4f61-a39a-cf4eebf2f669\") " pod="openshift-monitoring/node-exporter-b9rvb"
Jul 07 11:32:10.857165 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.856764    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfmjn\" (UniqueName: \"kubernetes.io/projected/f8e91d6e-7866-4576-bae6-a351d600d703-kube-api-access-wfmjn\") pod \"kube-state-metrics-7c9dbb8b7d-j4488\" (UID: \"f8e91d6e-7866-4576-bae6-a351d600d703\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j4488"
Jul 07 11:32:10.857165 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.856820    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f8e91d6e-7866-4576-bae6-a351d600d703-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-j4488\" (UID: \"f8e91d6e-7866-4576-bae6-a351d600d703\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j4488"
Jul 07 11:32:10.857165 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.856846    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/f8e91d6e-7866-4576-bae6-a351d600d703-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-j4488\" (UID: \"f8e91d6e-7866-4576-bae6-a351d600d703\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j4488"
Jul 07 11:32:10.857165 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.856889    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/54a5035f-302e-4f61-a39a-cf4eebf2f669-node-exporter-wtmp\") pod \"node-exporter-b9rvb\" (UID: \"54a5035f-302e-4f61-a39a-cf4eebf2f669\") " pod="openshift-monitoring/node-exporter-b9rvb"
Jul 07 11:32:10.857429 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.857375    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/54a5035f-302e-4f61-a39a-cf4eebf2f669-node-exporter-accelerators-collector-config\") pod \"node-exporter-b9rvb\" (UID: \"54a5035f-302e-4f61-a39a-cf4eebf2f669\") " pod="openshift-monitoring/node-exporter-b9rvb"
Jul 07 11:32:10.857490 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.857439    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/54a5035f-302e-4f61-a39a-cf4eebf2f669-sys\") pod \"node-exporter-b9rvb\" (UID: \"54a5035f-302e-4f61-a39a-cf4eebf2f669\") " pod="openshift-monitoring/node-exporter-b9rvb"
Jul 07 11:32:10.857599 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.857553    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/f8e91d6e-7866-4576-bae6-a351d600d703-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-j4488\" (UID: \"f8e91d6e-7866-4576-bae6-a351d600d703\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j4488"
Jul 07 11:32:10.857798 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.857759    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/54a5035f-302e-4f61-a39a-cf4eebf2f669-metrics-client-ca\") pod \"node-exporter-b9rvb\" (UID: \"54a5035f-302e-4f61-a39a-cf4eebf2f669\") " pod="openshift-monitoring/node-exporter-b9rvb"
Jul 07 11:32:10.857965 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.857943    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/54a5035f-302e-4f61-a39a-cf4eebf2f669-node-exporter-textfile\") pod \"node-exporter-b9rvb\" (UID: \"54a5035f-302e-4f61-a39a-cf4eebf2f669\") " pod="openshift-monitoring/node-exporter-b9rvb"
Jul 07 11:32:10.858873 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.858849    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f8e91d6e-7866-4576-bae6-a351d600d703-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-j4488\" (UID: \"f8e91d6e-7866-4576-bae6-a351d600d703\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j4488"
Jul 07 11:32:10.859286 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.859263    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/54a5035f-302e-4f61-a39a-cf4eebf2f669-node-exporter-tls\") pod \"node-exporter-b9rvb\" (UID: \"54a5035f-302e-4f61-a39a-cf4eebf2f669\") " pod="openshift-monitoring/node-exporter-b9rvb"
Jul 07 11:32:10.859869 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.859848    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/54a5035f-302e-4f61-a39a-cf4eebf2f669-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b9rvb\" (UID: \"54a5035f-302e-4f61-a39a-cf4eebf2f669\") " pod="openshift-monitoring/node-exporter-b9rvb"
Jul 07 11:32:10.860067 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.860044    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f8e91d6e-7866-4576-bae6-a351d600d703-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-j4488\" (UID: \"f8e91d6e-7866-4576-bae6-a351d600d703\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j4488"
Jul 07 11:32:10.863851 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.863818    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-74d5b\" (UniqueName: \"kubernetes.io/projected/54a5035f-302e-4f61-a39a-cf4eebf2f669-kube-api-access-74d5b\") pod \"node-exporter-b9rvb\" (UID: \"54a5035f-302e-4f61-a39a-cf4eebf2f669\") " pod="openshift-monitoring/node-exporter-b9rvb"
Jul 07 11:32:10.865418 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.865399    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfmjn\" (UniqueName: \"kubernetes.io/projected/f8e91d6e-7866-4576-bae6-a351d600d703-kube-api-access-wfmjn\") pod \"kube-state-metrics-7c9dbb8b7d-j4488\" (UID: \"f8e91d6e-7866-4576-bae6-a351d600d703\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j4488"
Jul 07 11:32:10.897199 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.897136    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r4q9n" event={"ID":"1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32","Type":"ContainerStarted","Data":"faf37b895fabc49bebc5d47b6fce67eee90a4086add3b160562c06aecb1b415a"}
Jul 07 11:32:10.916427 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.916365    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-r4q9n" podStartSLOduration=33.664652186 podStartE2EDuration="40.91634591s" totalImagesPullingTime="7.251693724s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:31:30 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:32:10.915591106 +0000 UTC m=+318.889231521" watchObservedRunningTime="2026-07-07 11:32:10.91634591 +0000 UTC m=+318.889986320"
Jul 07 11:32:10.953533 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.953502    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j4488"
Jul 07 11:32:10.975409 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.975315    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-gwvbz"]
Jul 07 11:32:10.976322 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:10.976051    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-b9rvb"
Jul 07 11:32:10.981856 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:32:10.981822    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e9e44ad_ca0e_4ae8_af94_0d51e0259ce2.slice/crio-c1892ca3f5d15a297084891a0774b9ecd8374a84c09b32872a92d7e65f08227b WatchSource:0}: Error finding container c1892ca3f5d15a297084891a0774b9ecd8374a84c09b32872a92d7e65f08227b: Status 404 returned error can't find the container with id c1892ca3f5d15a297084891a0774b9ecd8374a84c09b32872a92d7e65f08227b
Jul 07 11:32:10.997704 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:32:10.997333    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54a5035f_302e_4f61_a39a_cf4eebf2f669.slice/crio-d56b246ad98b19c809f33c507aa9bbc3b1250a6f51c53796da656defe7d96e59 WatchSource:0}: Error finding container d56b246ad98b19c809f33c507aa9bbc3b1250a6f51c53796da656defe7d96e59: Status 404 returned error can't find the container with id d56b246ad98b19c809f33c507aa9bbc3b1250a6f51c53796da656defe7d96e59
Jul 07 11:32:11.115273 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:11.115249    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j4488"]
Jul 07 11:32:11.117460 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:32:11.117431    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8e91d6e_7866_4576_bae6_a351d600d703.slice/crio-cd717f5e6df581a9bd806d38971c828d3133879a8eddeaa7049199e9d535f10a WatchSource:0}: Error finding container cd717f5e6df581a9bd806d38971c828d3133879a8eddeaa7049199e9d535f10a: Status 404 returned error can't find the container with id cd717f5e6df581a9bd806d38971c828d3133879a8eddeaa7049199e9d535f10a
Jul 07 11:32:11.903690 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:11.903599    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b9rvb" event={"ID":"54a5035f-302e-4f61-a39a-cf4eebf2f669","Type":"ContainerStarted","Data":"d56b246ad98b19c809f33c507aa9bbc3b1250a6f51c53796da656defe7d96e59"}
Jul 07 11:32:11.905865 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:11.905744    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j4488" event={"ID":"f8e91d6e-7866-4576-bae6-a351d600d703","Type":"ContainerStarted","Data":"cd717f5e6df581a9bd806d38971c828d3133879a8eddeaa7049199e9d535f10a"}
Jul 07 11:32:11.909417 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:11.908928    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-gwvbz" event={"ID":"3e9e44ad-ca0e-4ae8-af94-0d51e0259ce2","Type":"ContainerStarted","Data":"9fc480b70ce38587ff8cfab6da01a0e59cf7ef2ad4ec9d4c7a9910551ea62516"}
Jul 07 11:32:11.909417 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:11.908987    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-gwvbz" event={"ID":"3e9e44ad-ca0e-4ae8-af94-0d51e0259ce2","Type":"ContainerStarted","Data":"725271ebd9d76fb41deb661d45a3aa8640e3bb82fde87c21a471d3a277a8b2b4"}
Jul 07 11:32:11.909417 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:11.909017    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-gwvbz" event={"ID":"3e9e44ad-ca0e-4ae8-af94-0d51e0259ce2","Type":"ContainerStarted","Data":"c1892ca3f5d15a297084891a0774b9ecd8374a84c09b32872a92d7e65f08227b"}
Jul 07 11:32:13.918300 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:13.918261    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-gwvbz" event={"ID":"3e9e44ad-ca0e-4ae8-af94-0d51e0259ce2","Type":"ContainerStarted","Data":"f244a32d9b9c9f44d7830aa2fc821314758b46d2ce12640617ccf29b6a2e2c74"}
Jul 07 11:32:13.919732 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:13.919706    2570 generic.go:356] "Generic (PLEG): container finished" podID="54a5035f-302e-4f61-a39a-cf4eebf2f669" containerID="c857fd4c5ee66e447d4652eea1e183ddc1401161e13d6f302df27e226940b8e7" exitCode=0
Jul 07 11:32:13.919870 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:13.919787    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b9rvb" event={"ID":"54a5035f-302e-4f61-a39a-cf4eebf2f669","Type":"ContainerDied","Data":"c857fd4c5ee66e447d4652eea1e183ddc1401161e13d6f302df27e226940b8e7"}
Jul 07 11:32:13.921666 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:13.921646    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j4488" event={"ID":"f8e91d6e-7866-4576-bae6-a351d600d703","Type":"ContainerStarted","Data":"bb91adf68c71823a4bec31ffeeedf7f75ca72ff7332c9f0b3d30f4ebf15ea908"}
Jul 07 11:32:13.921783 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:13.921694    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j4488" event={"ID":"f8e91d6e-7866-4576-bae6-a351d600d703","Type":"ContainerStarted","Data":"dcab7a892af3f6b4283a6a64a2422a8f1de2da1c1f6b0708e601426d14b9c252"}
Jul 07 11:32:13.921783 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:13.921728    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j4488" event={"ID":"f8e91d6e-7866-4576-bae6-a351d600d703","Type":"ContainerStarted","Data":"5f3946706d52022a12905ae9a6d6bd22d008c64b76ec9a5ccd7650ba9599e95a"}
Jul 07 11:32:13.943885 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:13.943840    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-gwvbz" podStartSLOduration=2.244070443 podStartE2EDuration="3.943819428s" totalImagesPullingTime="1.699748985s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:32:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:32:13.942865072 +0000 UTC m=+321.916505482" watchObservedRunningTime="2026-07-07 11:32:13.943819428 +0000 UTC m=+321.917459837"
Jul 07 11:32:13.964979 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:13.964922    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j4488" podStartSLOduration=2.25374756 podStartE2EDuration="3.964900473s" totalImagesPullingTime="1.711152913s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:32:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:32:13.964568618 +0000 UTC m=+321.938209027" watchObservedRunningTime="2026-07-07 11:32:13.964900473 +0000 UTC m=+321.938540883"
Jul 07 11:32:14.926880 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:14.926842    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b9rvb" event={"ID":"54a5035f-302e-4f61-a39a-cf4eebf2f669","Type":"ContainerStarted","Data":"8d9e8d20ad170febc1ebbf8a49763c705d561290d39dcbdc7f0f9e52ec630042"}
Jul 07 11:32:14.927288 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:14.926898    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b9rvb" event={"ID":"54a5035f-302e-4f61-a39a-cf4eebf2f669","Type":"ContainerStarted","Data":"fb9531843a90bedd73018f3b37b786bea48ea132fdbef8e3e7a29c3034cde6f7"}
Jul 07 11:32:14.947392 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:14.947340    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-b9rvb" podStartSLOduration=3.019449565 podStartE2EDuration="4.947326232s" totalImagesPullingTime="1.827283136s" totalInitContainerRuntime="100.593531ms" isStatefulPod=true podCreationTimestamp="2026-07-07 11:32:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:32:14.946064871 +0000 UTC m=+322.919705317" watchObservedRunningTime="2026-07-07 11:32:14.947326232 +0000 UTC m=+322.920966642"
Jul 07 11:32:16.483260 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:16.483204    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-696d746fbb-8plv4"]
Jul 07 11:32:16.483624 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:32:16.483543    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-696d746fbb-8plv4" podUID="1aa5a3e9-5a49-4837-8fd2-389682ccef44"
Jul 07 11:32:16.933477 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:16.933447    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-696d746fbb-8plv4"
Jul 07 11:32:16.940812 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:16.940782    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-696d746fbb-8plv4"
Jul 07 11:32:17.021488 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:17.021453    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1aa5a3e9-5a49-4837-8fd2-389682ccef44-trusted-ca\") pod \"1aa5a3e9-5a49-4837-8fd2-389682ccef44\" (UID: \"1aa5a3e9-5a49-4837-8fd2-389682ccef44\") "
Jul 07 11:32:17.021656 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:17.021512    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1aa5a3e9-5a49-4837-8fd2-389682ccef44-ca-trust-extracted\") pod \"1aa5a3e9-5a49-4837-8fd2-389682ccef44\" (UID: \"1aa5a3e9-5a49-4837-8fd2-389682ccef44\") "
Jul 07 11:32:17.021656 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:17.021547    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7v7sh\" (UniqueName: \"kubernetes.io/projected/1aa5a3e9-5a49-4837-8fd2-389682ccef44-kube-api-access-7v7sh\") pod \"1aa5a3e9-5a49-4837-8fd2-389682ccef44\" (UID: \"1aa5a3e9-5a49-4837-8fd2-389682ccef44\") "
Jul 07 11:32:17.021656 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:17.021575    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1aa5a3e9-5a49-4837-8fd2-389682ccef44-image-registry-private-configuration\") pod \"1aa5a3e9-5a49-4837-8fd2-389682ccef44\" (UID: \"1aa5a3e9-5a49-4837-8fd2-389682ccef44\") "
Jul 07 11:32:17.021656 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:17.021603    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1aa5a3e9-5a49-4837-8fd2-389682ccef44-installation-pull-secrets\") pod \"1aa5a3e9-5a49-4837-8fd2-389682ccef44\" (UID: \"1aa5a3e9-5a49-4837-8fd2-389682ccef44\") "
Jul 07 11:32:17.021656 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:17.021628    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1aa5a3e9-5a49-4837-8fd2-389682ccef44-registry-certificates\") pod \"1aa5a3e9-5a49-4837-8fd2-389682ccef44\" (UID: \"1aa5a3e9-5a49-4837-8fd2-389682ccef44\") "
Jul 07 11:32:17.021877 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:17.021750    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aa5a3e9-5a49-4837-8fd2-389682ccef44-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "1aa5a3e9-5a49-4837-8fd2-389682ccef44" (UID: "1aa5a3e9-5a49-4837-8fd2-389682ccef44"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:32:17.021877 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:17.021870    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1aa5a3e9-5a49-4837-8fd2-389682ccef44-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "1aa5a3e9-5a49-4837-8fd2-389682ccef44" (UID: "1aa5a3e9-5a49-4837-8fd2-389682ccef44"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 11:32:17.021977 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:17.021955    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1aa5a3e9-5a49-4837-8fd2-389682ccef44-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "1aa5a3e9-5a49-4837-8fd2-389682ccef44" (UID: "1aa5a3e9-5a49-4837-8fd2-389682ccef44"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 11:32:17.022034 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:17.021991    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1aa5a3e9-5a49-4837-8fd2-389682ccef44-bound-sa-token\") pod \"1aa5a3e9-5a49-4837-8fd2-389682ccef44\" (UID: \"1aa5a3e9-5a49-4837-8fd2-389682ccef44\") "
Jul 07 11:32:17.022370 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:17.022346    2570 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1aa5a3e9-5a49-4837-8fd2-389682ccef44-registry-certificates\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:32:17.022551 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:17.022528    2570 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1aa5a3e9-5a49-4837-8fd2-389682ccef44-trusted-ca\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:32:17.022650 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:17.022556    2570 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1aa5a3e9-5a49-4837-8fd2-389682ccef44-ca-trust-extracted\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:32:17.023987 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:17.023960    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1aa5a3e9-5a49-4837-8fd2-389682ccef44-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "1aa5a3e9-5a49-4837-8fd2-389682ccef44" (UID: "1aa5a3e9-5a49-4837-8fd2-389682ccef44"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:32:17.024061 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:17.023986    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1aa5a3e9-5a49-4837-8fd2-389682ccef44-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "1aa5a3e9-5a49-4837-8fd2-389682ccef44" (UID: "1aa5a3e9-5a49-4837-8fd2-389682ccef44"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 11:32:17.024061 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:17.024038    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1aa5a3e9-5a49-4837-8fd2-389682ccef44-kube-api-access-7v7sh" (OuterVolumeSpecName: "kube-api-access-7v7sh") pod "1aa5a3e9-5a49-4837-8fd2-389682ccef44" (UID: "1aa5a3e9-5a49-4837-8fd2-389682ccef44"). InnerVolumeSpecName "kube-api-access-7v7sh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:32:17.024142 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:17.024052    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1aa5a3e9-5a49-4837-8fd2-389682ccef44-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "1aa5a3e9-5a49-4837-8fd2-389682ccef44" (UID: "1aa5a3e9-5a49-4837-8fd2-389682ccef44"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 11:32:17.123725 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:17.123676    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7v7sh\" (UniqueName: \"kubernetes.io/projected/1aa5a3e9-5a49-4837-8fd2-389682ccef44-kube-api-access-7v7sh\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:32:17.123962 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:17.123945    2570 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1aa5a3e9-5a49-4837-8fd2-389682ccef44-image-registry-private-configuration\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:32:17.124071 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:17.124059    2570 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1aa5a3e9-5a49-4837-8fd2-389682ccef44-installation-pull-secrets\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:32:17.124199 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:17.124187    2570 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1aa5a3e9-5a49-4837-8fd2-389682ccef44-bound-sa-token\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:32:17.936403 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:17.936374    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-696d746fbb-8plv4"
Jul 07 11:32:17.980776 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:17.980746    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-696d746fbb-8plv4"]
Jul 07 11:32:17.984179 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:17.984152    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-696d746fbb-8plv4"]
Jul 07 11:32:18.032504 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:18.032468    2570 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1aa5a3e9-5a49-4837-8fd2-389682ccef44-registry-tls\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:32:19.940026 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:32:19.939990    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1aa5a3e9-5a49-4837-8fd2-389682ccef44" path="/var/lib/kubelet/pods/1aa5a3e9-5a49-4837-8fd2-389682ccef44/volumes"
Jul 07 11:33:05.113110 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:33:05.113016    2570 generic.go:356] "Generic (PLEG): container finished" podID="49b59d5c-4db1-4013-8d2e-ac8690c4723c" containerID="0de11e630a5e85f9835184cccbdbcc99f55fd402d346b8f7d5cfef4cea8df141" exitCode=0
Jul 07 11:33:05.113110 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:33:05.113088    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-ngjjc" event={"ID":"49b59d5c-4db1-4013-8d2e-ac8690c4723c","Type":"ContainerDied","Data":"0de11e630a5e85f9835184cccbdbcc99f55fd402d346b8f7d5cfef4cea8df141"}
Jul 07 11:33:05.113631 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:33:05.113448    2570 scope.go:117] "RemoveContainer" containerID="0de11e630a5e85f9835184cccbdbcc99f55fd402d346b8f7d5cfef4cea8df141"
Jul 07 11:33:06.118018 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:33:06.117986    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-ngjjc" event={"ID":"49b59d5c-4db1-4013-8d2e-ac8690c4723c","Type":"ContainerStarted","Data":"3334ab79194d7175793b7c06cc1660982924fd0dd2bfb5f79bd839676e1a0e22"}
Jul 07 11:33:07.966225 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:33:07.966179    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f75e11d0-c0de-4a8d-849d-807a87aa1497-metrics-certs\") pod \"network-metrics-daemon-ppmbz\" (UID: \"f75e11d0-c0de-4a8d-849d-807a87aa1497\") " pod="openshift-multus/network-metrics-daemon-ppmbz"
Jul 07 11:33:07.968682 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:33:07.968654    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f75e11d0-c0de-4a8d-849d-807a87aa1497-metrics-certs\") pod \"network-metrics-daemon-ppmbz\" (UID: \"f75e11d0-c0de-4a8d-849d-807a87aa1497\") " pod="openshift-multus/network-metrics-daemon-ppmbz"
Jul 07 11:33:08.037170 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:33:08.037132    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-nfmsb\""
Jul 07 11:33:08.044377 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:33:08.044346    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ppmbz"
Jul 07 11:33:08.196165 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:33:08.195996    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ppmbz"]
Jul 07 11:33:08.198690 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:33:08.198655    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf75e11d0_c0de_4a8d_849d_807a87aa1497.slice/crio-d75b059c2c301b731172ffea35c7ed7f929709b334b3b91d26a0161c69cc3ddf WatchSource:0}: Error finding container d75b059c2c301b731172ffea35c7ed7f929709b334b3b91d26a0161c69cc3ddf: Status 404 returned error can't find the container with id d75b059c2c301b731172ffea35c7ed7f929709b334b3b91d26a0161c69cc3ddf
Jul 07 11:33:09.137859 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:33:09.137825    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ppmbz" event={"ID":"f75e11d0-c0de-4a8d-849d-807a87aa1497","Type":"ContainerStarted","Data":"d75b059c2c301b731172ffea35c7ed7f929709b334b3b91d26a0161c69cc3ddf"}
Jul 07 11:33:10.143778 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:33:10.143742    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ppmbz" event={"ID":"f75e11d0-c0de-4a8d-849d-807a87aa1497","Type":"ContainerStarted","Data":"bb9cee8eb783a956b45a846b1d4b13bd9313955be30df45dd92d088d743e128e"}
Jul 07 11:33:10.144282 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:33:10.143795    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ppmbz" event={"ID":"f75e11d0-c0de-4a8d-849d-807a87aa1497","Type":"ContainerStarted","Data":"4d2f14133dc09f1e669b1e5feb5c73ab148b0334c6f16c3d534a45c6090c2196"}
Jul 07 11:33:10.160556 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:33:10.160493    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-ppmbz" podStartSLOduration=375.051118265 podStartE2EDuration="6m16.160473131s" totalImagesPullingTime="1.109354866s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:26:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:33:10.159561323 +0000 UTC m=+378.133201744" watchObservedRunningTime="2026-07-07 11:33:10.160473131 +0000 UTC m=+378.134113540"
Jul 07 11:33:38.768175 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:33:38.768129    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6bffdf9778-wlk99" podUID="88a181af-4d7c-4398-a53b-69ab8f2e4035"
Jul 07 11:33:38.768175 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:33:38.768138    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-9gqzt" podUID="bfcc5fcf-866f-405b-ae62-dcf401627bc7"
Jul 07 11:33:39.249136 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:33:39.249098    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9gqzt"
Jul 07 11:33:39.249459 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:33:39.249099    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-wlk99"
Jul 07 11:33:41.349038 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:33:41.348993    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bfcc5fcf-866f-405b-ae62-dcf401627bc7-metrics-tls\") pod \"dns-default-9gqzt\" (UID: \"bfcc5fcf-866f-405b-ae62-dcf401627bc7\") " pod="openshift-dns/dns-default-9gqzt"
Jul 07 11:33:41.349479 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:33:41.349168    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/88a181af-4d7c-4398-a53b-69ab8f2e4035-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-wlk99\" (UID: \"88a181af-4d7c-4398-a53b-69ab8f2e4035\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-wlk99"
Jul 07 11:33:41.351459 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:33:41.351428    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bfcc5fcf-866f-405b-ae62-dcf401627bc7-metrics-tls\") pod \"dns-default-9gqzt\" (UID: \"bfcc5fcf-866f-405b-ae62-dcf401627bc7\") " pod="openshift-dns/dns-default-9gqzt"
Jul 07 11:33:41.351976 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:33:41.351950    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/88a181af-4d7c-4398-a53b-69ab8f2e4035-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-wlk99\" (UID: \"88a181af-4d7c-4398-a53b-69ab8f2e4035\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-wlk99"
Jul 07 11:33:41.653096 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:33:41.653009    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-vgmpj\""
Jul 07 11:33:41.654134 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:33:41.654118    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xqnk4\""
Jul 07 11:33:41.660303 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:33:41.660283    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-wlk99"
Jul 07 11:33:41.660407 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:33:41.660378    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9gqzt"
Jul 07 11:33:41.804651 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:33:41.804622    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9gqzt"]
Jul 07 11:33:41.807889 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:33:41.807859    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfcc5fcf_866f_405b_ae62_dcf401627bc7.slice/crio-b85987c00c00b845e68cef43b8aa4390cfc55c24e10fba32c11970339ad11ea8 WatchSource:0}: Error finding container b85987c00c00b845e68cef43b8aa4390cfc55c24e10fba32c11970339ad11ea8: Status 404 returned error can't find the container with id b85987c00c00b845e68cef43b8aa4390cfc55c24e10fba32c11970339ad11ea8
Jul 07 11:33:41.818842 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:33:41.818819    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-wlk99"]
Jul 07 11:33:41.820906 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:33:41.820882    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88a181af_4d7c_4398_a53b_69ab8f2e4035.slice/crio-7b61834433de0f643a1b73cb6a4dd2dc731cdf3b512828bd2c40fd0930efb21c WatchSource:0}: Error finding container 7b61834433de0f643a1b73cb6a4dd2dc731cdf3b512828bd2c40fd0930efb21c: Status 404 returned error can't find the container with id 7b61834433de0f643a1b73cb6a4dd2dc731cdf3b512828bd2c40fd0930efb21c
Jul 07 11:33:42.262104 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:33:42.262065    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-wlk99" event={"ID":"88a181af-4d7c-4398-a53b-69ab8f2e4035","Type":"ContainerStarted","Data":"7b61834433de0f643a1b73cb6a4dd2dc731cdf3b512828bd2c40fd0930efb21c"}
Jul 07 11:33:42.263297 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:33:42.263270    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9gqzt" event={"ID":"bfcc5fcf-866f-405b-ae62-dcf401627bc7","Type":"ContainerStarted","Data":"b85987c00c00b845e68cef43b8aa4390cfc55c24e10fba32c11970339ad11ea8"}
Jul 07 11:33:44.275477 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:33:44.275425    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-wlk99" event={"ID":"88a181af-4d7c-4398-a53b-69ab8f2e4035","Type":"ContainerStarted","Data":"008615e7d5fe6915c955124801ba69b624132f4fb1d676944cb51db0d927d801"}
Jul 07 11:33:44.277228 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:33:44.277188    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9gqzt" event={"ID":"bfcc5fcf-866f-405b-ae62-dcf401627bc7","Type":"ContainerStarted","Data":"0821d9525474f83b241611f8e06063ddb732f5c4ce29cff3e0fbb7c650f9070d"}
Jul 07 11:33:44.277352 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:33:44.277252    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9gqzt" event={"ID":"bfcc5fcf-866f-405b-ae62-dcf401627bc7","Type":"ContainerStarted","Data":"5e3c8e61e37240ef4042232f7133952d346b4bd77e55e0f388cc115577b263ec"}
Jul 07 11:33:44.277352 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:33:44.277278    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-9gqzt"
Jul 07 11:33:44.294347 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:33:44.294289    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-wlk99" podStartSLOduration=406.743236094 podStartE2EDuration="6m48.294267657s" totalImagesPullingTime="1.551031563s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:26:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:33:44.293820181 +0000 UTC m=+412.267460603" watchObservedRunningTime="2026-07-07 11:33:44.294267657 +0000 UTC m=+412.267908069"
Jul 07 11:33:44.316125 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:33:44.316074    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-9gqzt" podStartSLOduration=373.704919347 podStartE2EDuration="6m15.316054641s" totalImagesPullingTime="1.611135294s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:27:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:33:44.314511427 +0000 UTC m=+412.288151838" watchObservedRunningTime="2026-07-07 11:33:44.316054641 +0000 UTC m=+412.289695060"
Jul 07 11:33:54.283246 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:33:54.283199    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-9gqzt"
Jul 07 11:35:02.162402 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:02.162366    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m7bl"]
Jul 07 11:35:02.165649 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:02.165630    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m7bl"
Jul 07 11:35:02.168423 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:02.168400    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 07 11:35:02.168567 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:02.168475    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 07 11:35:02.169772 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:02.169752    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-cntgm\""
Jul 07 11:35:02.175820 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:02.175796    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m7bl"]
Jul 07 11:35:02.176280 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:02.176262    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e543544c-e201-4af0-957b-2605d0431bd9-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m7bl\" (UID: \"e543544c-e201-4af0-957b-2605d0431bd9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m7bl"
Jul 07 11:35:02.176350 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:02.176293    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e543544c-e201-4af0-957b-2605d0431bd9-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m7bl\" (UID: \"e543544c-e201-4af0-957b-2605d0431bd9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m7bl"
Jul 07 11:35:02.176350 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:02.176337    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vh859\" (UniqueName: \"kubernetes.io/projected/e543544c-e201-4af0-957b-2605d0431bd9-kube-api-access-vh859\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m7bl\" (UID: \"e543544c-e201-4af0-957b-2605d0431bd9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m7bl"
Jul 07 11:35:02.277498 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:02.277457    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e543544c-e201-4af0-957b-2605d0431bd9-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m7bl\" (UID: \"e543544c-e201-4af0-957b-2605d0431bd9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m7bl"
Jul 07 11:35:02.277498 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:02.277500    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e543544c-e201-4af0-957b-2605d0431bd9-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m7bl\" (UID: \"e543544c-e201-4af0-957b-2605d0431bd9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m7bl"
Jul 07 11:35:02.277730 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:02.277542    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vh859\" (UniqueName: \"kubernetes.io/projected/e543544c-e201-4af0-957b-2605d0431bd9-kube-api-access-vh859\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m7bl\" (UID: \"e543544c-e201-4af0-957b-2605d0431bd9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m7bl"
Jul 07 11:35:02.277882 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:02.277863    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e543544c-e201-4af0-957b-2605d0431bd9-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m7bl\" (UID: \"e543544c-e201-4af0-957b-2605d0431bd9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m7bl"
Jul 07 11:35:02.277922 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:02.277884    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e543544c-e201-4af0-957b-2605d0431bd9-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m7bl\" (UID: \"e543544c-e201-4af0-957b-2605d0431bd9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m7bl"
Jul 07 11:35:02.286622 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:02.286596    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vh859\" (UniqueName: \"kubernetes.io/projected/e543544c-e201-4af0-957b-2605d0431bd9-kube-api-access-vh859\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m7bl\" (UID: \"e543544c-e201-4af0-957b-2605d0431bd9\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m7bl"
Jul 07 11:35:02.475652 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:02.475614    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m7bl"
Jul 07 11:35:02.618239 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:02.618192    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m7bl"]
Jul 07 11:35:02.621306 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:35:02.621270    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode543544c_e201_4af0_957b_2605d0431bd9.slice/crio-86bf028958213c2805652e7e69a4446a430a791264d10f9c477e10722d09e0ce WatchSource:0}: Error finding container 86bf028958213c2805652e7e69a4446a430a791264d10f9c477e10722d09e0ce: Status 404 returned error can't find the container with id 86bf028958213c2805652e7e69a4446a430a791264d10f9c477e10722d09e0ce
Jul 07 11:35:03.577390 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:03.577358    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m7bl" event={"ID":"e543544c-e201-4af0-957b-2605d0431bd9","Type":"ContainerStarted","Data":"86bf028958213c2805652e7e69a4446a430a791264d10f9c477e10722d09e0ce"}
Jul 07 11:35:09.601952 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:09.601911    2570 generic.go:356] "Generic (PLEG): container finished" podID="e543544c-e201-4af0-957b-2605d0431bd9" containerID="189e27387d7355f9ba5f58680fb23a6b444b3e57796ae53840a785f762df9acb" exitCode=0
Jul 07 11:35:09.602373 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:09.601988    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m7bl" event={"ID":"e543544c-e201-4af0-957b-2605d0431bd9","Type":"ContainerDied","Data":"189e27387d7355f9ba5f58680fb23a6b444b3e57796ae53840a785f762df9acb"}
Jul 07 11:35:12.617756 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:12.617723    2570 generic.go:356] "Generic (PLEG): container finished" podID="e543544c-e201-4af0-957b-2605d0431bd9" containerID="c2b49af432a6e2acb2925024ac1fc2139a6f41941a3de60011cac8ce0b4f1c13" exitCode=0
Jul 07 11:35:12.618135 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:12.617762    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m7bl" event={"ID":"e543544c-e201-4af0-957b-2605d0431bd9","Type":"ContainerDied","Data":"c2b49af432a6e2acb2925024ac1fc2139a6f41941a3de60011cac8ce0b4f1c13"}
Jul 07 11:35:21.659933 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:21.659895    2570 generic.go:356] "Generic (PLEG): container finished" podID="e543544c-e201-4af0-957b-2605d0431bd9" containerID="7411a6b1cf7176a2ed73979faf78dc270be0db0cc0c1f54540d070b109cd8d4f" exitCode=0
Jul 07 11:35:21.660410 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:21.659939    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m7bl" event={"ID":"e543544c-e201-4af0-957b-2605d0431bd9","Type":"ContainerDied","Data":"7411a6b1cf7176a2ed73979faf78dc270be0db0cc0c1f54540d070b109cd8d4f"}
Jul 07 11:35:22.792389 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:22.792364    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m7bl"
Jul 07 11:35:22.849971 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:22.849941    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vh859\" (UniqueName: \"kubernetes.io/projected/e543544c-e201-4af0-957b-2605d0431bd9-kube-api-access-vh859\") pod \"e543544c-e201-4af0-957b-2605d0431bd9\" (UID: \"e543544c-e201-4af0-957b-2605d0431bd9\") "
Jul 07 11:35:22.850138 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:22.850039    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e543544c-e201-4af0-957b-2605d0431bd9-util\") pod \"e543544c-e201-4af0-957b-2605d0431bd9\" (UID: \"e543544c-e201-4af0-957b-2605d0431bd9\") "
Jul 07 11:35:22.850138 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:22.850092    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e543544c-e201-4af0-957b-2605d0431bd9-bundle\") pod \"e543544c-e201-4af0-957b-2605d0431bd9\" (UID: \"e543544c-e201-4af0-957b-2605d0431bd9\") "
Jul 07 11:35:22.850774 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:22.850740    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e543544c-e201-4af0-957b-2605d0431bd9-bundle" (OuterVolumeSpecName: "bundle") pod "e543544c-e201-4af0-957b-2605d0431bd9" (UID: "e543544c-e201-4af0-957b-2605d0431bd9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:35:22.852191 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:22.852167    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e543544c-e201-4af0-957b-2605d0431bd9-kube-api-access-vh859" (OuterVolumeSpecName: "kube-api-access-vh859") pod "e543544c-e201-4af0-957b-2605d0431bd9" (UID: "e543544c-e201-4af0-957b-2605d0431bd9"). InnerVolumeSpecName "kube-api-access-vh859". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:35:22.854934 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:22.854913    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e543544c-e201-4af0-957b-2605d0431bd9-util" (OuterVolumeSpecName: "util") pod "e543544c-e201-4af0-957b-2605d0431bd9" (UID: "e543544c-e201-4af0-957b-2605d0431bd9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:35:22.951275 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:22.951242    2570 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e543544c-e201-4af0-957b-2605d0431bd9-util\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:35:22.951275 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:22.951273    2570 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e543544c-e201-4af0-957b-2605d0431bd9-bundle\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:35:22.951275 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:22.951284    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vh859\" (UniqueName: \"kubernetes.io/projected/e543544c-e201-4af0-957b-2605d0431bd9-kube-api-access-vh859\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:35:23.669018 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:23.668987    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m7bl"
Jul 07 11:35:23.669018 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:23.668991    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405m7bl" event={"ID":"e543544c-e201-4af0-957b-2605d0431bd9","Type":"ContainerDied","Data":"86bf028958213c2805652e7e69a4446a430a791264d10f9c477e10722d09e0ce"}
Jul 07 11:35:23.669311 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:23.669067    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="86bf028958213c2805652e7e69a4446a430a791264d10f9c477e10722d09e0ce"
Jul 07 11:35:34.324231 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:34.324178    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-kpl5v"]
Jul 07 11:35:34.324625 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:34.324469    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e543544c-e201-4af0-957b-2605d0431bd9" containerName="util"
Jul 07 11:35:34.324625 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:34.324480    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="e543544c-e201-4af0-957b-2605d0431bd9" containerName="util"
Jul 07 11:35:34.324625 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:34.324498    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e543544c-e201-4af0-957b-2605d0431bd9" containerName="pull"
Jul 07 11:35:34.324625 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:34.324504    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="e543544c-e201-4af0-957b-2605d0431bd9" containerName="pull"
Jul 07 11:35:34.324625 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:34.324511    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e543544c-e201-4af0-957b-2605d0431bd9" containerName="extract"
Jul 07 11:35:34.324625 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:34.324518    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="e543544c-e201-4af0-957b-2605d0431bd9" containerName="extract"
Jul 07 11:35:34.324625 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:34.324562    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="e543544c-e201-4af0-957b-2605d0431bd9" containerName="extract"
Jul 07 11:35:34.327511 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:34.327495    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-kpl5v"
Jul 07 11:35:34.330446 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:34.330418    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 07 11:35:34.331733 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:34.331716    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 07 11:35:34.331855 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:34.331719    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-czm47\""
Jul 07 11:35:34.342428 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:34.342403    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-kpl5v"]
Jul 07 11:35:34.448899 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:34.448853    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4fmp\" (UniqueName: \"kubernetes.io/projected/c6b3aa9f-7a8b-4272-a44e-c2cb1dcbf7cc-kube-api-access-g4fmp\") pod \"cert-manager-webhook-6486ffd997-kpl5v\" (UID: \"c6b3aa9f-7a8b-4272-a44e-c2cb1dcbf7cc\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kpl5v"
Jul 07 11:35:34.449083 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:34.448961    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c6b3aa9f-7a8b-4272-a44e-c2cb1dcbf7cc-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-kpl5v\" (UID: \"c6b3aa9f-7a8b-4272-a44e-c2cb1dcbf7cc\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kpl5v"
Jul 07 11:35:34.550009 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:34.549970    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c6b3aa9f-7a8b-4272-a44e-c2cb1dcbf7cc-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-kpl5v\" (UID: \"c6b3aa9f-7a8b-4272-a44e-c2cb1dcbf7cc\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kpl5v"
Jul 07 11:35:34.550185 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:34.550055    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4fmp\" (UniqueName: \"kubernetes.io/projected/c6b3aa9f-7a8b-4272-a44e-c2cb1dcbf7cc-kube-api-access-g4fmp\") pod \"cert-manager-webhook-6486ffd997-kpl5v\" (UID: \"c6b3aa9f-7a8b-4272-a44e-c2cb1dcbf7cc\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kpl5v"
Jul 07 11:35:34.561701 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:34.561663    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c6b3aa9f-7a8b-4272-a44e-c2cb1dcbf7cc-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-kpl5v\" (UID: \"c6b3aa9f-7a8b-4272-a44e-c2cb1dcbf7cc\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kpl5v"
Jul 07 11:35:34.564073 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:34.564049    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4fmp\" (UniqueName: \"kubernetes.io/projected/c6b3aa9f-7a8b-4272-a44e-c2cb1dcbf7cc-kube-api-access-g4fmp\") pod \"cert-manager-webhook-6486ffd997-kpl5v\" (UID: \"c6b3aa9f-7a8b-4272-a44e-c2cb1dcbf7cc\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kpl5v"
Jul 07 11:35:34.636069 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:34.635986    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-kpl5v"
Jul 07 11:35:34.779493 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:34.779464    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-kpl5v"]
Jul 07 11:35:34.781530 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:35:34.781503    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc6b3aa9f_7a8b_4272_a44e_c2cb1dcbf7cc.slice/crio-8911e17a8383ba413a8d2d4478fab9098311dc05889824d520d4203c55bf21a1 WatchSource:0}: Error finding container 8911e17a8383ba413a8d2d4478fab9098311dc05889824d520d4203c55bf21a1: Status 404 returned error can't find the container with id 8911e17a8383ba413a8d2d4478fab9098311dc05889824d520d4203c55bf21a1
Jul 07 11:35:35.465292 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:35.465251    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvxwf"]
Jul 07 11:35:35.469929 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:35.469903    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvxwf"
Jul 07 11:35:35.473494 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:35.473466    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-cntgm\""
Jul 07 11:35:35.473636 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:35.473565    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 07 11:35:35.474821 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:35.474798    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 07 11:35:35.479813 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:35.479778    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvxwf"]
Jul 07 11:35:35.558720 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:35.558680    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0b45d03b-38c1-49d1-9e7d-4263a4246534-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvxwf\" (UID: \"0b45d03b-38c1-49d1-9e7d-4263a4246534\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvxwf"
Jul 07 11:35:35.558935 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:35.558747    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkg7v\" (UniqueName: \"kubernetes.io/projected/0b45d03b-38c1-49d1-9e7d-4263a4246534-kube-api-access-gkg7v\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvxwf\" (UID: \"0b45d03b-38c1-49d1-9e7d-4263a4246534\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvxwf"
Jul 07 11:35:35.558935 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:35.558821    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0b45d03b-38c1-49d1-9e7d-4263a4246534-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvxwf\" (UID: \"0b45d03b-38c1-49d1-9e7d-4263a4246534\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvxwf"
Jul 07 11:35:35.659426 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:35.659389    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0b45d03b-38c1-49d1-9e7d-4263a4246534-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvxwf\" (UID: \"0b45d03b-38c1-49d1-9e7d-4263a4246534\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvxwf"
Jul 07 11:35:35.659622 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:35.659459    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkg7v\" (UniqueName: \"kubernetes.io/projected/0b45d03b-38c1-49d1-9e7d-4263a4246534-kube-api-access-gkg7v\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvxwf\" (UID: \"0b45d03b-38c1-49d1-9e7d-4263a4246534\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvxwf"
Jul 07 11:35:35.659622 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:35.659504    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0b45d03b-38c1-49d1-9e7d-4263a4246534-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvxwf\" (UID: \"0b45d03b-38c1-49d1-9e7d-4263a4246534\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvxwf"
Jul 07 11:35:35.659881 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:35.659861    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0b45d03b-38c1-49d1-9e7d-4263a4246534-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvxwf\" (UID: \"0b45d03b-38c1-49d1-9e7d-4263a4246534\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvxwf"
Jul 07 11:35:35.659941 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:35.659860    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0b45d03b-38c1-49d1-9e7d-4263a4246534-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvxwf\" (UID: \"0b45d03b-38c1-49d1-9e7d-4263a4246534\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvxwf"
Jul 07 11:35:35.673137 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:35.673100    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkg7v\" (UniqueName: \"kubernetes.io/projected/0b45d03b-38c1-49d1-9e7d-4263a4246534-kube-api-access-gkg7v\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvxwf\" (UID: \"0b45d03b-38c1-49d1-9e7d-4263a4246534\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvxwf"
Jul 07 11:35:35.723924 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:35.723841    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-kpl5v" event={"ID":"c6b3aa9f-7a8b-4272-a44e-c2cb1dcbf7cc","Type":"ContainerStarted","Data":"8911e17a8383ba413a8d2d4478fab9098311dc05889824d520d4203c55bf21a1"}
Jul 07 11:35:35.780868 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:35.780833    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvxwf"
Jul 07 11:35:35.932178 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:35.932142    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvxwf"]
Jul 07 11:35:35.935557 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:35:35.935523    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b45d03b_38c1_49d1_9e7d_4263a4246534.slice/crio-ec7a3ee935bcf147e0703ee9bf1f8db2999012272e0dddddae7d7c9d44a3d92b WatchSource:0}: Error finding container ec7a3ee935bcf147e0703ee9bf1f8db2999012272e0dddddae7d7c9d44a3d92b: Status 404 returned error can't find the container with id ec7a3ee935bcf147e0703ee9bf1f8db2999012272e0dddddae7d7c9d44a3d92b
Jul 07 11:35:36.730284 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:36.730205    2570 generic.go:356] "Generic (PLEG): container finished" podID="0b45d03b-38c1-49d1-9e7d-4263a4246534" containerID="968dd5fb938a482f83cf2b0e414f6e876f85bdd1dac81453b1826fdf0f744ac3" exitCode=0
Jul 07 11:35:36.730742 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:36.730311    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvxwf" event={"ID":"0b45d03b-38c1-49d1-9e7d-4263a4246534","Type":"ContainerDied","Data":"968dd5fb938a482f83cf2b0e414f6e876f85bdd1dac81453b1826fdf0f744ac3"}
Jul 07 11:35:36.730742 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:36.730376    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvxwf" event={"ID":"0b45d03b-38c1-49d1-9e7d-4263a4246534","Type":"ContainerStarted","Data":"ec7a3ee935bcf147e0703ee9bf1f8db2999012272e0dddddae7d7c9d44a3d92b"}
Jul 07 11:35:37.737934 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:37.737894    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-kpl5v" event={"ID":"c6b3aa9f-7a8b-4272-a44e-c2cb1dcbf7cc","Type":"ContainerStarted","Data":"4c46260df5fb181fff1b4a560d1d447b31e230c2d33492c21eaa10f549301476"}
Jul 07 11:35:37.738428 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:37.738037    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-kpl5v"
Jul 07 11:35:39.748178 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:39.748143    2570 generic.go:356] "Generic (PLEG): container finished" podID="0b45d03b-38c1-49d1-9e7d-4263a4246534" containerID="c769b7f1acabdc8cf538100f35f317926aca9a972c8a309bb7af326cd651f49d" exitCode=0
Jul 07 11:35:39.748709 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:39.748248    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvxwf" event={"ID":"0b45d03b-38c1-49d1-9e7d-4263a4246534","Type":"ContainerDied","Data":"c769b7f1acabdc8cf538100f35f317926aca9a972c8a309bb7af326cd651f49d"}
Jul 07 11:35:39.768775 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:39.768725    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-kpl5v" podStartSLOduration=3.003234071 podStartE2EDuration="5.768707241s" totalImagesPullingTime="2.76547317s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:35:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:35:37.754404545 +0000 UTC m=+525.728044957" watchObservedRunningTime="2026-07-07 11:35:39.768707241 +0000 UTC m=+527.742347677"
Jul 07 11:35:40.754890 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:40.754857    2570 generic.go:356] "Generic (PLEG): container finished" podID="0b45d03b-38c1-49d1-9e7d-4263a4246534" containerID="9628a0644682929679cb6ac61021f0cf4954f66ec16efc6459ba222ba95611a1" exitCode=0
Jul 07 11:35:40.755306 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:40.754939    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvxwf" event={"ID":"0b45d03b-38c1-49d1-9e7d-4263a4246534","Type":"ContainerDied","Data":"9628a0644682929679cb6ac61021f0cf4954f66ec16efc6459ba222ba95611a1"}
Jul 07 11:35:41.915782 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:41.915748    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvxwf"
Jul 07 11:35:42.013776 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:42.013686    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0b45d03b-38c1-49d1-9e7d-4263a4246534-bundle\") pod \"0b45d03b-38c1-49d1-9e7d-4263a4246534\" (UID: \"0b45d03b-38c1-49d1-9e7d-4263a4246534\") "
Jul 07 11:35:42.013776 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:42.013759    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0b45d03b-38c1-49d1-9e7d-4263a4246534-util\") pod \"0b45d03b-38c1-49d1-9e7d-4263a4246534\" (UID: \"0b45d03b-38c1-49d1-9e7d-4263a4246534\") "
Jul 07 11:35:42.014000 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:42.013800    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gkg7v\" (UniqueName: \"kubernetes.io/projected/0b45d03b-38c1-49d1-9e7d-4263a4246534-kube-api-access-gkg7v\") pod \"0b45d03b-38c1-49d1-9e7d-4263a4246534\" (UID: \"0b45d03b-38c1-49d1-9e7d-4263a4246534\") "
Jul 07 11:35:42.014124 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:42.014095    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b45d03b-38c1-49d1-9e7d-4263a4246534-bundle" (OuterVolumeSpecName: "bundle") pod "0b45d03b-38c1-49d1-9e7d-4263a4246534" (UID: "0b45d03b-38c1-49d1-9e7d-4263a4246534"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:35:42.015956 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:42.015925    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b45d03b-38c1-49d1-9e7d-4263a4246534-kube-api-access-gkg7v" (OuterVolumeSpecName: "kube-api-access-gkg7v") pod "0b45d03b-38c1-49d1-9e7d-4263a4246534" (UID: "0b45d03b-38c1-49d1-9e7d-4263a4246534"). InnerVolumeSpecName "kube-api-access-gkg7v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:35:42.017957 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:42.017919    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b45d03b-38c1-49d1-9e7d-4263a4246534-util" (OuterVolumeSpecName: "util") pod "0b45d03b-38c1-49d1-9e7d-4263a4246534" (UID: "0b45d03b-38c1-49d1-9e7d-4263a4246534"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:35:42.114481 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:42.114444    2570 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0b45d03b-38c1-49d1-9e7d-4263a4246534-bundle\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:35:42.114481 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:42.114475    2570 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0b45d03b-38c1-49d1-9e7d-4263a4246534-util\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:35:42.114481 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:42.114484    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gkg7v\" (UniqueName: \"kubernetes.io/projected/0b45d03b-38c1-49d1-9e7d-4263a4246534-kube-api-access-gkg7v\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:35:42.764140 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:42.764112    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvxwf"
Jul 07 11:35:42.764140 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:42.764127    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvxwf" event={"ID":"0b45d03b-38c1-49d1-9e7d-4263a4246534","Type":"ContainerDied","Data":"ec7a3ee935bcf147e0703ee9bf1f8db2999012272e0dddddae7d7c9d44a3d92b"}
Jul 07 11:35:42.764372 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:42.764167    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ec7a3ee935bcf147e0703ee9bf1f8db2999012272e0dddddae7d7c9d44a3d92b"
Jul 07 11:35:43.745262 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:43.745200    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-kpl5v"
Jul 07 11:35:45.392105 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:45.392069    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-9fvdr"]
Jul 07 11:35:45.392516 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:45.392433    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b45d03b-38c1-49d1-9e7d-4263a4246534" containerName="pull"
Jul 07 11:35:45.392516 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:45.392446    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b45d03b-38c1-49d1-9e7d-4263a4246534" containerName="pull"
Jul 07 11:35:45.392516 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:45.392467    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b45d03b-38c1-49d1-9e7d-4263a4246534" containerName="util"
Jul 07 11:35:45.392516 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:45.392472    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b45d03b-38c1-49d1-9e7d-4263a4246534" containerName="util"
Jul 07 11:35:45.392516 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:45.392480    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b45d03b-38c1-49d1-9e7d-4263a4246534" containerName="extract"
Jul 07 11:35:45.392516 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:45.392486    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b45d03b-38c1-49d1-9e7d-4263a4246534" containerName="extract"
Jul 07 11:35:45.392801 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:45.392543    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b45d03b-38c1-49d1-9e7d-4263a4246534" containerName="extract"
Jul 07 11:35:45.395371 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:45.395354    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-9fvdr"
Jul 07 11:35:45.398253 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:45.398227    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-kvjmt\""
Jul 07 11:35:45.405816 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:45.405789    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-9fvdr"]
Jul 07 11:35:45.541864 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:45.541828    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwfq8\" (UniqueName: \"kubernetes.io/projected/e136f2ba-2c28-4717-8983-445d559dc657-kube-api-access-pwfq8\") pod \"cert-manager-55b4d954d5-9fvdr\" (UID: \"e136f2ba-2c28-4717-8983-445d559dc657\") " pod="cert-manager/cert-manager-55b4d954d5-9fvdr"
Jul 07 11:35:45.542064 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:45.541899    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e136f2ba-2c28-4717-8983-445d559dc657-bound-sa-token\") pod \"cert-manager-55b4d954d5-9fvdr\" (UID: \"e136f2ba-2c28-4717-8983-445d559dc657\") " pod="cert-manager/cert-manager-55b4d954d5-9fvdr"
Jul 07 11:35:45.642825 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:45.642739    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwfq8\" (UniqueName: \"kubernetes.io/projected/e136f2ba-2c28-4717-8983-445d559dc657-kube-api-access-pwfq8\") pod \"cert-manager-55b4d954d5-9fvdr\" (UID: \"e136f2ba-2c28-4717-8983-445d559dc657\") " pod="cert-manager/cert-manager-55b4d954d5-9fvdr"
Jul 07 11:35:45.642825 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:45.642819    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e136f2ba-2c28-4717-8983-445d559dc657-bound-sa-token\") pod \"cert-manager-55b4d954d5-9fvdr\" (UID: \"e136f2ba-2c28-4717-8983-445d559dc657\") " pod="cert-manager/cert-manager-55b4d954d5-9fvdr"
Jul 07 11:35:45.651483 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:45.651455    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e136f2ba-2c28-4717-8983-445d559dc657-bound-sa-token\") pod \"cert-manager-55b4d954d5-9fvdr\" (UID: \"e136f2ba-2c28-4717-8983-445d559dc657\") " pod="cert-manager/cert-manager-55b4d954d5-9fvdr"
Jul 07 11:35:45.651647 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:45.651545    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwfq8\" (UniqueName: \"kubernetes.io/projected/e136f2ba-2c28-4717-8983-445d559dc657-kube-api-access-pwfq8\") pod \"cert-manager-55b4d954d5-9fvdr\" (UID: \"e136f2ba-2c28-4717-8983-445d559dc657\") " pod="cert-manager/cert-manager-55b4d954d5-9fvdr"
Jul 07 11:35:45.704106 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:45.704072    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-9fvdr"
Jul 07 11:35:45.834920 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:45.834897    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-9fvdr"]
Jul 07 11:35:45.837205 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:35:45.837179    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode136f2ba_2c28_4717_8983_445d559dc657.slice/crio-f03177516ba7f35155d17d38d5d63cef688e2934fce94a4f9338ec719285fd9a WatchSource:0}: Error finding container f03177516ba7f35155d17d38d5d63cef688e2934fce94a4f9338ec719285fd9a: Status 404 returned error can't find the container with id f03177516ba7f35155d17d38d5d63cef688e2934fce94a4f9338ec719285fd9a
Jul 07 11:35:46.782324 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:46.782283    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-9fvdr" event={"ID":"e136f2ba-2c28-4717-8983-445d559dc657","Type":"ContainerStarted","Data":"a01d7a142878273af0ee2430103d2da8b250cb4abcbc0bc8ca7c378824d9fd2c"}
Jul 07 11:35:46.782728 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:46.782335    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-9fvdr" event={"ID":"e136f2ba-2c28-4717-8983-445d559dc657","Type":"ContainerStarted","Data":"f03177516ba7f35155d17d38d5d63cef688e2934fce94a4f9338ec719285fd9a"}
Jul 07 11:35:46.801735 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:46.801679    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-9fvdr" podStartSLOduration=1.8016635029999999 podStartE2EDuration="1.801663503s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:35:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:35:46.800506757 +0000 UTC m=+534.774147168" watchObservedRunningTime="2026-07-07 11:35:46.801663503 +0000 UTC m=+534.775303912"
Jul 07 11:35:54.796690 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:54.796658    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr5fd"]
Jul 07 11:35:54.823113 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:54.823080    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr5fd"]
Jul 07 11:35:54.823287 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:54.823182    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr5fd"
Jul 07 11:35:54.826300 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:54.826278    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 07 11:35:54.827546 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:54.827526    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 07 11:35:54.827658 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:54.827561    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-cntgm\""
Jul 07 11:35:54.918266 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:54.918231    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6fddcc5b-fa11-48bb-ad56-cc7341c567d8-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr5fd\" (UID: \"6fddcc5b-fa11-48bb-ad56-cc7341c567d8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr5fd"
Jul 07 11:35:54.918452 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:54.918274    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwv82\" (UniqueName: \"kubernetes.io/projected/6fddcc5b-fa11-48bb-ad56-cc7341c567d8-kube-api-access-qwv82\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr5fd\" (UID: \"6fddcc5b-fa11-48bb-ad56-cc7341c567d8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr5fd"
Jul 07 11:35:54.918452 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:54.918312    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6fddcc5b-fa11-48bb-ad56-cc7341c567d8-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr5fd\" (UID: \"6fddcc5b-fa11-48bb-ad56-cc7341c567d8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr5fd"
Jul 07 11:35:55.019275 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:55.019239    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6fddcc5b-fa11-48bb-ad56-cc7341c567d8-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr5fd\" (UID: \"6fddcc5b-fa11-48bb-ad56-cc7341c567d8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr5fd"
Jul 07 11:35:55.019480 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:55.019291    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwv82\" (UniqueName: \"kubernetes.io/projected/6fddcc5b-fa11-48bb-ad56-cc7341c567d8-kube-api-access-qwv82\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr5fd\" (UID: \"6fddcc5b-fa11-48bb-ad56-cc7341c567d8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr5fd"
Jul 07 11:35:55.019480 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:55.019357    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6fddcc5b-fa11-48bb-ad56-cc7341c567d8-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr5fd\" (UID: \"6fddcc5b-fa11-48bb-ad56-cc7341c567d8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr5fd"
Jul 07 11:35:55.019678 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:55.019652    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6fddcc5b-fa11-48bb-ad56-cc7341c567d8-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr5fd\" (UID: \"6fddcc5b-fa11-48bb-ad56-cc7341c567d8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr5fd"
Jul 07 11:35:55.019749 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:55.019696    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6fddcc5b-fa11-48bb-ad56-cc7341c567d8-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr5fd\" (UID: \"6fddcc5b-fa11-48bb-ad56-cc7341c567d8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr5fd"
Jul 07 11:35:55.028545 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:55.028514    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwv82\" (UniqueName: \"kubernetes.io/projected/6fddcc5b-fa11-48bb-ad56-cc7341c567d8-kube-api-access-qwv82\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr5fd\" (UID: \"6fddcc5b-fa11-48bb-ad56-cc7341c567d8\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr5fd"
Jul 07 11:35:55.132737 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:55.132646    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr5fd"
Jul 07 11:35:55.282932 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:55.282906    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr5fd"]
Jul 07 11:35:55.285590 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:35:55.285560    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fddcc5b_fa11_48bb_ad56_cc7341c567d8.slice/crio-8b4168ad49fa18a036ee48cc85a6e7a3ef4ff5f192bf9df240477e7c9b98e8cd WatchSource:0}: Error finding container 8b4168ad49fa18a036ee48cc85a6e7a3ef4ff5f192bf9df240477e7c9b98e8cd: Status 404 returned error can't find the container with id 8b4168ad49fa18a036ee48cc85a6e7a3ef4ff5f192bf9df240477e7c9b98e8cd
Jul 07 11:35:55.819925 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:55.819889    2570 generic.go:356] "Generic (PLEG): container finished" podID="6fddcc5b-fa11-48bb-ad56-cc7341c567d8" containerID="7dc192e275201165b8f412b1dea5c9ed191768f600f4796fd13caadb57ef129b" exitCode=0
Jul 07 11:35:55.820497 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:55.819937    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr5fd" event={"ID":"6fddcc5b-fa11-48bb-ad56-cc7341c567d8","Type":"ContainerDied","Data":"7dc192e275201165b8f412b1dea5c9ed191768f600f4796fd13caadb57ef129b"}
Jul 07 11:35:55.820497 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:55.819975    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr5fd" event={"ID":"6fddcc5b-fa11-48bb-ad56-cc7341c567d8","Type":"ContainerStarted","Data":"8b4168ad49fa18a036ee48cc85a6e7a3ef4ff5f192bf9df240477e7c9b98e8cd"}
Jul 07 11:35:56.826016 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:56.825921    2570 generic.go:356] "Generic (PLEG): container finished" podID="6fddcc5b-fa11-48bb-ad56-cc7341c567d8" containerID="777392ea8f165a6dfcd0c448dfb6fa77a378fb5c83607098f10057e85588d57a" exitCode=0
Jul 07 11:35:56.826016 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:56.825975    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr5fd" event={"ID":"6fddcc5b-fa11-48bb-ad56-cc7341c567d8","Type":"ContainerDied","Data":"777392ea8f165a6dfcd0c448dfb6fa77a378fb5c83607098f10057e85588d57a"}
Jul 07 11:35:57.832509 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:57.832470    2570 generic.go:356] "Generic (PLEG): container finished" podID="6fddcc5b-fa11-48bb-ad56-cc7341c567d8" containerID="f58328d557804588ed3911f0ad3f2faf964af44c7c41ea30ad7fadf59a041d9c" exitCode=0
Jul 07 11:35:57.832906 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:57.832520    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr5fd" event={"ID":"6fddcc5b-fa11-48bb-ad56-cc7341c567d8","Type":"ContainerDied","Data":"f58328d557804588ed3911f0ad3f2faf964af44c7c41ea30ad7fadf59a041d9c"}
Jul 07 11:35:58.969418 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:58.969391    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr5fd"
Jul 07 11:35:59.051652 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:59.051621    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6fddcc5b-fa11-48bb-ad56-cc7341c567d8-util\") pod \"6fddcc5b-fa11-48bb-ad56-cc7341c567d8\" (UID: \"6fddcc5b-fa11-48bb-ad56-cc7341c567d8\") "
Jul 07 11:35:59.051861 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:59.051660    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qwv82\" (UniqueName: \"kubernetes.io/projected/6fddcc5b-fa11-48bb-ad56-cc7341c567d8-kube-api-access-qwv82\") pod \"6fddcc5b-fa11-48bb-ad56-cc7341c567d8\" (UID: \"6fddcc5b-fa11-48bb-ad56-cc7341c567d8\") "
Jul 07 11:35:59.051861 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:59.051732    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6fddcc5b-fa11-48bb-ad56-cc7341c567d8-bundle\") pod \"6fddcc5b-fa11-48bb-ad56-cc7341c567d8\" (UID: \"6fddcc5b-fa11-48bb-ad56-cc7341c567d8\") "
Jul 07 11:35:59.052599 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:59.052554    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fddcc5b-fa11-48bb-ad56-cc7341c567d8-bundle" (OuterVolumeSpecName: "bundle") pod "6fddcc5b-fa11-48bb-ad56-cc7341c567d8" (UID: "6fddcc5b-fa11-48bb-ad56-cc7341c567d8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:35:59.053849 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:59.053816    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fddcc5b-fa11-48bb-ad56-cc7341c567d8-kube-api-access-qwv82" (OuterVolumeSpecName: "kube-api-access-qwv82") pod "6fddcc5b-fa11-48bb-ad56-cc7341c567d8" (UID: "6fddcc5b-fa11-48bb-ad56-cc7341c567d8"). InnerVolumeSpecName "kube-api-access-qwv82". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:35:59.058247 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:59.058160    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fddcc5b-fa11-48bb-ad56-cc7341c567d8-util" (OuterVolumeSpecName: "util") pod "6fddcc5b-fa11-48bb-ad56-cc7341c567d8" (UID: "6fddcc5b-fa11-48bb-ad56-cc7341c567d8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:35:59.152691 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:59.152602    2570 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6fddcc5b-fa11-48bb-ad56-cc7341c567d8-util\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:35:59.152691 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:59.152633    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qwv82\" (UniqueName: \"kubernetes.io/projected/6fddcc5b-fa11-48bb-ad56-cc7341c567d8-kube-api-access-qwv82\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:35:59.152691 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:59.152643    2570 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6fddcc5b-fa11-48bb-ad56-cc7341c567d8-bundle\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:35:59.842958 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:59.842927    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr5fd"
Jul 07 11:35:59.842958 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:59.842949    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqr5fd" event={"ID":"6fddcc5b-fa11-48bb-ad56-cc7341c567d8","Type":"ContainerDied","Data":"8b4168ad49fa18a036ee48cc85a6e7a3ef4ff5f192bf9df240477e7c9b98e8cd"}
Jul 07 11:35:59.843194 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:35:59.842995    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8b4168ad49fa18a036ee48cc85a6e7a3ef4ff5f192bf9df240477e7c9b98e8cd"
Jul 07 11:36:12.180026 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:12.179990    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-f68d9489b-zlljb"]
Jul 07 11:36:12.180638 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:12.180496    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fddcc5b-fa11-48bb-ad56-cc7341c567d8" containerName="util"
Jul 07 11:36:12.180638 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:12.180516    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fddcc5b-fa11-48bb-ad56-cc7341c567d8" containerName="util"
Jul 07 11:36:12.180638 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:12.180528    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fddcc5b-fa11-48bb-ad56-cc7341c567d8" containerName="pull"
Jul 07 11:36:12.180638 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:12.180536    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fddcc5b-fa11-48bb-ad56-cc7341c567d8" containerName="pull"
Jul 07 11:36:12.180638 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:12.180547    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fddcc5b-fa11-48bb-ad56-cc7341c567d8" containerName="extract"
Jul 07 11:36:12.180638 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:12.180567    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fddcc5b-fa11-48bb-ad56-cc7341c567d8" containerName="extract"
Jul 07 11:36:12.180638 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:12.180639    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fddcc5b-fa11-48bb-ad56-cc7341c567d8" containerName="extract"
Jul 07 11:36:12.184941 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:12.184921    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-f68d9489b-zlljb"
Jul 07 11:36:12.188695 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:12.188657    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-29df8\""
Jul 07 11:36:12.188904 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:12.188716    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 07 11:36:12.189496 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:12.189478    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 07 11:36:12.190476 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:12.190454    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 07 11:36:12.193912 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:12.193894    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 07 11:36:12.212617 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:12.212591    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-f68d9489b-zlljb"]
Jul 07 11:36:12.260105 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:12.260073    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c245d200-66ef-4446-809a-db8b19cb4b77-manifests\") pod \"opendatahub-operator-controller-manager-f68d9489b-zlljb\" (UID: \"c245d200-66ef-4446-809a-db8b19cb4b77\") " pod="opendatahub/opendatahub-operator-controller-manager-f68d9489b-zlljb"
Jul 07 11:36:12.260105 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:12.260108    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89b2l\" (UniqueName: \"kubernetes.io/projected/c245d200-66ef-4446-809a-db8b19cb4b77-kube-api-access-89b2l\") pod \"opendatahub-operator-controller-manager-f68d9489b-zlljb\" (UID: \"c245d200-66ef-4446-809a-db8b19cb4b77\") " pod="opendatahub/opendatahub-operator-controller-manager-f68d9489b-zlljb"
Jul 07 11:36:12.260368 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:12.260136    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c245d200-66ef-4446-809a-db8b19cb4b77-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f68d9489b-zlljb\" (UID: \"c245d200-66ef-4446-809a-db8b19cb4b77\") " pod="opendatahub/opendatahub-operator-controller-manager-f68d9489b-zlljb"
Jul 07 11:36:12.260368 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:12.260253    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c245d200-66ef-4446-809a-db8b19cb4b77-webhook-cert\") pod \"opendatahub-operator-controller-manager-f68d9489b-zlljb\" (UID: \"c245d200-66ef-4446-809a-db8b19cb4b77\") " pod="opendatahub/opendatahub-operator-controller-manager-f68d9489b-zlljb"
Jul 07 11:36:12.260368 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:12.260287    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c245d200-66ef-4446-809a-db8b19cb4b77-tmp\") pod \"opendatahub-operator-controller-manager-f68d9489b-zlljb\" (UID: \"c245d200-66ef-4446-809a-db8b19cb4b77\") " pod="opendatahub/opendatahub-operator-controller-manager-f68d9489b-zlljb"
Jul 07 11:36:12.361132 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:12.361095    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c245d200-66ef-4446-809a-db8b19cb4b77-manifests\") pod \"opendatahub-operator-controller-manager-f68d9489b-zlljb\" (UID: \"c245d200-66ef-4446-809a-db8b19cb4b77\") " pod="opendatahub/opendatahub-operator-controller-manager-f68d9489b-zlljb"
Jul 07 11:36:12.361132 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:12.361132    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89b2l\" (UniqueName: \"kubernetes.io/projected/c245d200-66ef-4446-809a-db8b19cb4b77-kube-api-access-89b2l\") pod \"opendatahub-operator-controller-manager-f68d9489b-zlljb\" (UID: \"c245d200-66ef-4446-809a-db8b19cb4b77\") " pod="opendatahub/opendatahub-operator-controller-manager-f68d9489b-zlljb"
Jul 07 11:36:12.361417 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:12.361156    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c245d200-66ef-4446-809a-db8b19cb4b77-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f68d9489b-zlljb\" (UID: \"c245d200-66ef-4446-809a-db8b19cb4b77\") " pod="opendatahub/opendatahub-operator-controller-manager-f68d9489b-zlljb"
Jul 07 11:36:12.361417 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:12.361193    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c245d200-66ef-4446-809a-db8b19cb4b77-webhook-cert\") pod \"opendatahub-operator-controller-manager-f68d9489b-zlljb\" (UID: \"c245d200-66ef-4446-809a-db8b19cb4b77\") " pod="opendatahub/opendatahub-operator-controller-manager-f68d9489b-zlljb"
Jul 07 11:36:12.361417 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:12.361247    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c245d200-66ef-4446-809a-db8b19cb4b77-tmp\") pod \"opendatahub-operator-controller-manager-f68d9489b-zlljb\" (UID: \"c245d200-66ef-4446-809a-db8b19cb4b77\") " pod="opendatahub/opendatahub-operator-controller-manager-f68d9489b-zlljb"
Jul 07 11:36:12.361581 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:12.361500    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c245d200-66ef-4446-809a-db8b19cb4b77-manifests\") pod \"opendatahub-operator-controller-manager-f68d9489b-zlljb\" (UID: \"c245d200-66ef-4446-809a-db8b19cb4b77\") " pod="opendatahub/opendatahub-operator-controller-manager-f68d9489b-zlljb"
Jul 07 11:36:12.363451 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:12.363429    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c245d200-66ef-4446-809a-db8b19cb4b77-tmp\") pod \"opendatahub-operator-controller-manager-f68d9489b-zlljb\" (UID: \"c245d200-66ef-4446-809a-db8b19cb4b77\") " pod="opendatahub/opendatahub-operator-controller-manager-f68d9489b-zlljb"
Jul 07 11:36:12.363668 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:12.363648    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c245d200-66ef-4446-809a-db8b19cb4b77-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f68d9489b-zlljb\" (UID: \"c245d200-66ef-4446-809a-db8b19cb4b77\") " pod="opendatahub/opendatahub-operator-controller-manager-f68d9489b-zlljb"
Jul 07 11:36:12.363790 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:12.363773    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c245d200-66ef-4446-809a-db8b19cb4b77-webhook-cert\") pod \"opendatahub-operator-controller-manager-f68d9489b-zlljb\" (UID: \"c245d200-66ef-4446-809a-db8b19cb4b77\") " pod="opendatahub/opendatahub-operator-controller-manager-f68d9489b-zlljb"
Jul 07 11:36:12.369415 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:12.369394    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-89b2l\" (UniqueName: \"kubernetes.io/projected/c245d200-66ef-4446-809a-db8b19cb4b77-kube-api-access-89b2l\") pod \"opendatahub-operator-controller-manager-f68d9489b-zlljb\" (UID: \"c245d200-66ef-4446-809a-db8b19cb4b77\") " pod="opendatahub/opendatahub-operator-controller-manager-f68d9489b-zlljb"
Jul 07 11:36:12.497296 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:12.497258    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-f68d9489b-zlljb"
Jul 07 11:36:12.638829 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:12.638804    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-f68d9489b-zlljb"]
Jul 07 11:36:12.641130 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:36:12.641093    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc245d200_66ef_4446_809a_db8b19cb4b77.slice/crio-f49d0d79886d0e2a1d21bfaa3940b3563e27069ccf390bb6064830dabd93a29f WatchSource:0}: Error finding container f49d0d79886d0e2a1d21bfaa3940b3563e27069ccf390bb6064830dabd93a29f: Status 404 returned error can't find the container with id f49d0d79886d0e2a1d21bfaa3940b3563e27069ccf390bb6064830dabd93a29f
Jul 07 11:36:12.897922 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:12.897839    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f68d9489b-zlljb" event={"ID":"c245d200-66ef-4446-809a-db8b19cb4b77","Type":"ContainerStarted","Data":"f49d0d79886d0e2a1d21bfaa3940b3563e27069ccf390bb6064830dabd93a29f"}
Jul 07 11:36:15.915373 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:15.915333    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f68d9489b-zlljb" event={"ID":"c245d200-66ef-4446-809a-db8b19cb4b77","Type":"ContainerStarted","Data":"845a8c506ba4d667b346494ace4250b33db179df018ef0f61a955fd85d7eef2a"}
Jul 07 11:36:17.930186 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:17.930147    2570 generic.go:356] "Generic (PLEG): container finished" podID="c245d200-66ef-4446-809a-db8b19cb4b77" containerID="845a8c506ba4d667b346494ace4250b33db179df018ef0f61a955fd85d7eef2a" exitCode=0
Jul 07 11:36:17.930584 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:17.930226    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f68d9489b-zlljb" event={"ID":"c245d200-66ef-4446-809a-db8b19cb4b77","Type":"ContainerDied","Data":"845a8c506ba4d667b346494ace4250b33db179df018ef0f61a955fd85d7eef2a"}
Jul 07 11:36:18.936574 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:18.936530    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f68d9489b-zlljb" event={"ID":"c245d200-66ef-4446-809a-db8b19cb4b77","Type":"ContainerStarted","Data":"a3c12fb01fbf428001aaf958132b2276e2fbdb9fa52a8f56d7be917b375046bb"}
Jul 07 11:36:18.936951 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:18.936751    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-f68d9489b-zlljb"
Jul 07 11:36:18.960319 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:18.960267    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-f68d9489b-zlljb" podStartSLOduration=2.394328124 podStartE2EDuration="6.960253218s" totalImagesPullingTime="3.422490005s" totalInitContainerRuntime="1.143435089s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:36:12 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:36:18.95744867 +0000 UTC m=+566.931089122" watchObservedRunningTime="2026-07-07 11:36:18.960253218 +0000 UTC m=+566.933893647"
Jul 07 11:36:29.941937 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:29.941909    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-f68d9489b-zlljb"
Jul 07 11:36:32.070370 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:32.070329    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5474cb5f94-92m5v"]
Jul 07 11:36:32.078634 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:32.078602    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5474cb5f94-92m5v"
Jul 07 11:36:32.081596 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:32.081569    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 07 11:36:32.081728 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:32.081595    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-29r7s\""
Jul 07 11:36:32.081728 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:32.081600    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 07 11:36:32.093512 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:32.093486    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5474cb5f94-92m5v"]
Jul 07 11:36:32.231264 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:32.231231    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9cf22451-0c1c-4c2b-b2f4-c9dc6ff88f06-tmp\") pod \"kube-auth-proxy-5474cb5f94-92m5v\" (UID: \"9cf22451-0c1c-4c2b-b2f4-c9dc6ff88f06\") " pod="openshift-ingress/kube-auth-proxy-5474cb5f94-92m5v"
Jul 07 11:36:32.231446 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:32.231286    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9cf22451-0c1c-4c2b-b2f4-c9dc6ff88f06-tls-certs\") pod \"kube-auth-proxy-5474cb5f94-92m5v\" (UID: \"9cf22451-0c1c-4c2b-b2f4-c9dc6ff88f06\") " pod="openshift-ingress/kube-auth-proxy-5474cb5f94-92m5v"
Jul 07 11:36:32.231446 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:32.231316    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvjxb\" (UniqueName: \"kubernetes.io/projected/9cf22451-0c1c-4c2b-b2f4-c9dc6ff88f06-kube-api-access-nvjxb\") pod \"kube-auth-proxy-5474cb5f94-92m5v\" (UID: \"9cf22451-0c1c-4c2b-b2f4-c9dc6ff88f06\") " pod="openshift-ingress/kube-auth-proxy-5474cb5f94-92m5v"
Jul 07 11:36:32.332396 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:32.332300    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9cf22451-0c1c-4c2b-b2f4-c9dc6ff88f06-tmp\") pod \"kube-auth-proxy-5474cb5f94-92m5v\" (UID: \"9cf22451-0c1c-4c2b-b2f4-c9dc6ff88f06\") " pod="openshift-ingress/kube-auth-proxy-5474cb5f94-92m5v"
Jul 07 11:36:32.332396 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:32.332365    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9cf22451-0c1c-4c2b-b2f4-c9dc6ff88f06-tls-certs\") pod \"kube-auth-proxy-5474cb5f94-92m5v\" (UID: \"9cf22451-0c1c-4c2b-b2f4-c9dc6ff88f06\") " pod="openshift-ingress/kube-auth-proxy-5474cb5f94-92m5v"
Jul 07 11:36:32.332620 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:32.332397    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvjxb\" (UniqueName: \"kubernetes.io/projected/9cf22451-0c1c-4c2b-b2f4-c9dc6ff88f06-kube-api-access-nvjxb\") pod \"kube-auth-proxy-5474cb5f94-92m5v\" (UID: \"9cf22451-0c1c-4c2b-b2f4-c9dc6ff88f06\") " pod="openshift-ingress/kube-auth-proxy-5474cb5f94-92m5v"
Jul 07 11:36:32.332620 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:36:32.332467    2570 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 07 11:36:32.332620 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:36:32.332558    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9cf22451-0c1c-4c2b-b2f4-c9dc6ff88f06-tls-certs podName:9cf22451-0c1c-4c2b-b2f4-c9dc6ff88f06 nodeName:}" failed. No retries permitted until 2026-07-07 11:36:32.832536043 +0000 UTC m=+580.806176431 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/9cf22451-0c1c-4c2b-b2f4-c9dc6ff88f06-tls-certs") pod "kube-auth-proxy-5474cb5f94-92m5v" (UID: "9cf22451-0c1c-4c2b-b2f4-c9dc6ff88f06") : secret "kube-auth-proxy-tls" not found
Jul 07 11:36:32.334571 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:32.334548    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9cf22451-0c1c-4c2b-b2f4-c9dc6ff88f06-tmp\") pod \"kube-auth-proxy-5474cb5f94-92m5v\" (UID: \"9cf22451-0c1c-4c2b-b2f4-c9dc6ff88f06\") " pod="openshift-ingress/kube-auth-proxy-5474cb5f94-92m5v"
Jul 07 11:36:32.343833 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:32.343805    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvjxb\" (UniqueName: \"kubernetes.io/projected/9cf22451-0c1c-4c2b-b2f4-c9dc6ff88f06-kube-api-access-nvjxb\") pod \"kube-auth-proxy-5474cb5f94-92m5v\" (UID: \"9cf22451-0c1c-4c2b-b2f4-c9dc6ff88f06\") " pod="openshift-ingress/kube-auth-proxy-5474cb5f94-92m5v"
Jul 07 11:36:32.836930 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:32.836896    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9cf22451-0c1c-4c2b-b2f4-c9dc6ff88f06-tls-certs\") pod \"kube-auth-proxy-5474cb5f94-92m5v\" (UID: \"9cf22451-0c1c-4c2b-b2f4-c9dc6ff88f06\") " pod="openshift-ingress/kube-auth-proxy-5474cb5f94-92m5v"
Jul 07 11:36:32.839541 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:32.839516    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9cf22451-0c1c-4c2b-b2f4-c9dc6ff88f06-tls-certs\") pod \"kube-auth-proxy-5474cb5f94-92m5v\" (UID: \"9cf22451-0c1c-4c2b-b2f4-c9dc6ff88f06\") " pod="openshift-ingress/kube-auth-proxy-5474cb5f94-92m5v"
Jul 07 11:36:32.988692 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:32.988654    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5474cb5f94-92m5v"
Jul 07 11:36:33.134066 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:33.134043    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5474cb5f94-92m5v"]
Jul 07 11:36:33.136612 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:36:33.136567    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cf22451_0c1c_4c2b_b2f4_c9dc6ff88f06.slice/crio-c157b24d72ce08ce89eeeaaaf03da326b7dcca3bf7bdb5534fc57423f8be1ffc WatchSource:0}: Error finding container c157b24d72ce08ce89eeeaaaf03da326b7dcca3bf7bdb5534fc57423f8be1ffc: Status 404 returned error can't find the container with id c157b24d72ce08ce89eeeaaaf03da326b7dcca3bf7bdb5534fc57423f8be1ffc
Jul 07 11:36:33.146958 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:33.146931    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-7dl7x"]
Jul 07 11:36:33.151473 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:33.151454    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-7dl7x"
Jul 07 11:36:33.155073 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:33.155052    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 07 11:36:33.179010 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:33.178979    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-7dl7x"]
Jul 07 11:36:33.240293 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:33.240257    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb7560d9-3b8c-4d4b-99dd-190e3fbbd1b8-tls-certs\") pod \"model-serving-api-54d89bd5fc-7dl7x\" (UID: \"eb7560d9-3b8c-4d4b-99dd-190e3fbbd1b8\") " pod="opendatahub/model-serving-api-54d89bd5fc-7dl7x"
Jul 07 11:36:33.240472 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:33.240338    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8rp4\" (UniqueName: \"kubernetes.io/projected/eb7560d9-3b8c-4d4b-99dd-190e3fbbd1b8-kube-api-access-s8rp4\") pod \"model-serving-api-54d89bd5fc-7dl7x\" (UID: \"eb7560d9-3b8c-4d4b-99dd-190e3fbbd1b8\") " pod="opendatahub/model-serving-api-54d89bd5fc-7dl7x"
Jul 07 11:36:33.341738 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:33.341698    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8rp4\" (UniqueName: \"kubernetes.io/projected/eb7560d9-3b8c-4d4b-99dd-190e3fbbd1b8-kube-api-access-s8rp4\") pod \"model-serving-api-54d89bd5fc-7dl7x\" (UID: \"eb7560d9-3b8c-4d4b-99dd-190e3fbbd1b8\") " pod="opendatahub/model-serving-api-54d89bd5fc-7dl7x"
Jul 07 11:36:33.341888 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:33.341793    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb7560d9-3b8c-4d4b-99dd-190e3fbbd1b8-tls-certs\") pod \"model-serving-api-54d89bd5fc-7dl7x\" (UID: \"eb7560d9-3b8c-4d4b-99dd-190e3fbbd1b8\") " pod="opendatahub/model-serving-api-54d89bd5fc-7dl7x"
Jul 07 11:36:33.341933 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:36:33.341920    2570 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 07 11:36:33.341997 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:36:33.341987    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb7560d9-3b8c-4d4b-99dd-190e3fbbd1b8-tls-certs podName:eb7560d9-3b8c-4d4b-99dd-190e3fbbd1b8 nodeName:}" failed. No retries permitted until 2026-07-07 11:36:33.841965587 +0000 UTC m=+581.815605996 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/eb7560d9-3b8c-4d4b-99dd-190e3fbbd1b8-tls-certs") pod "model-serving-api-54d89bd5fc-7dl7x" (UID: "eb7560d9-3b8c-4d4b-99dd-190e3fbbd1b8") : secret "model-serving-api-tls" not found
Jul 07 11:36:33.357554 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:33.357523    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8rp4\" (UniqueName: \"kubernetes.io/projected/eb7560d9-3b8c-4d4b-99dd-190e3fbbd1b8-kube-api-access-s8rp4\") pod \"model-serving-api-54d89bd5fc-7dl7x\" (UID: \"eb7560d9-3b8c-4d4b-99dd-190e3fbbd1b8\") " pod="opendatahub/model-serving-api-54d89bd5fc-7dl7x"
Jul 07 11:36:33.846373 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:33.846330    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb7560d9-3b8c-4d4b-99dd-190e3fbbd1b8-tls-certs\") pod \"model-serving-api-54d89bd5fc-7dl7x\" (UID: \"eb7560d9-3b8c-4d4b-99dd-190e3fbbd1b8\") " pod="opendatahub/model-serving-api-54d89bd5fc-7dl7x"
Jul 07 11:36:33.846567 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:36:33.846499    2570 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 07 11:36:33.846567 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:36:33.846564    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb7560d9-3b8c-4d4b-99dd-190e3fbbd1b8-tls-certs podName:eb7560d9-3b8c-4d4b-99dd-190e3fbbd1b8 nodeName:}" failed. No retries permitted until 2026-07-07 11:36:34.846549279 +0000 UTC m=+582.820189667 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/eb7560d9-3b8c-4d4b-99dd-190e3fbbd1b8-tls-certs") pod "model-serving-api-54d89bd5fc-7dl7x" (UID: "eb7560d9-3b8c-4d4b-99dd-190e3fbbd1b8") : secret "model-serving-api-tls" not found
Jul 07 11:36:34.009048 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:34.009002    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5474cb5f94-92m5v" event={"ID":"9cf22451-0c1c-4c2b-b2f4-c9dc6ff88f06","Type":"ContainerStarted","Data":"c157b24d72ce08ce89eeeaaaf03da326b7dcca3bf7bdb5534fc57423f8be1ffc"}
Jul 07 11:36:34.856493 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:34.856452    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb7560d9-3b8c-4d4b-99dd-190e3fbbd1b8-tls-certs\") pod \"model-serving-api-54d89bd5fc-7dl7x\" (UID: \"eb7560d9-3b8c-4d4b-99dd-190e3fbbd1b8\") " pod="opendatahub/model-serving-api-54d89bd5fc-7dl7x"
Jul 07 11:36:34.858842 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:34.858822    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb7560d9-3b8c-4d4b-99dd-190e3fbbd1b8-tls-certs\") pod \"model-serving-api-54d89bd5fc-7dl7x\" (UID: \"eb7560d9-3b8c-4d4b-99dd-190e3fbbd1b8\") " pod="opendatahub/model-serving-api-54d89bd5fc-7dl7x"
Jul 07 11:36:34.963039 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:34.963003    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-7dl7x"
Jul 07 11:36:35.108754 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:35.108730    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-7dl7x"]
Jul 07 11:36:35.111393 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:36:35.111365    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb7560d9_3b8c_4d4b_99dd_190e3fbbd1b8.slice/crio-1b6e20b893e30322453d5c1a47b17681da25fda0b44ab1a2a9dd50ee1d0dfb16 WatchSource:0}: Error finding container 1b6e20b893e30322453d5c1a47b17681da25fda0b44ab1a2a9dd50ee1d0dfb16: Status 404 returned error can't find the container with id 1b6e20b893e30322453d5c1a47b17681da25fda0b44ab1a2a9dd50ee1d0dfb16
Jul 07 11:36:36.023032 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:36.022990    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-7dl7x" event={"ID":"eb7560d9-3b8c-4d4b-99dd-190e3fbbd1b8","Type":"ContainerStarted","Data":"1b6e20b893e30322453d5c1a47b17681da25fda0b44ab1a2a9dd50ee1d0dfb16"}
Jul 07 11:36:36.024801 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:36.024765    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5474cb5f94-92m5v" event={"ID":"9cf22451-0c1c-4c2b-b2f4-c9dc6ff88f06","Type":"ContainerStarted","Data":"595ff91929e1776edf87c39edb58d4b341a8ed68a4b9e79f9898fc2968df0690"}
Jul 07 11:36:36.046119 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:36.046058    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5474cb5f94-92m5v" podStartSLOduration=1.35997182 podStartE2EDuration="4.046039416s" totalImagesPullingTime="2.686067596s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:36:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:36:36.042636958 +0000 UTC m=+584.016277368" watchObservedRunningTime="2026-07-07 11:36:36.046039416 +0000 UTC m=+584.019679827"
Jul 07 11:36:38.036227 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:38.036173    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-7dl7x" event={"ID":"eb7560d9-3b8c-4d4b-99dd-190e3fbbd1b8","Type":"ContainerStarted","Data":"34ac80f82083bf863ff056126fe7763b4edd71c6944c47dea2a50a220b611860"}
Jul 07 11:36:38.058687 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:38.058631    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-7dl7x" podStartSLOduration=3.158115826 podStartE2EDuration="5.05861468s" totalImagesPullingTime="1.900498854s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:36:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:36:38.056272311 +0000 UTC m=+586.029912721" watchObservedRunningTime="2026-07-07 11:36:38.05861468 +0000 UTC m=+586.032255091"
Jul 07 11:36:39.041089 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:39.041048    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-7dl7x"
Jul 07 11:36:39.812272 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:39.812235    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48fwzr"]
Jul 07 11:36:39.816013 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:39.815995    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48fwzr"
Jul 07 11:36:39.819317 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:39.819300    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-5sphz\""
Jul 07 11:36:39.819391 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:39.819303    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 07 11:36:39.836844 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:39.836809    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48fwzr"]
Jul 07 11:36:39.899568 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:39.899527    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8abca133-c3a3-47eb-98d1-acaa6743e025-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48fwzr\" (UID: \"8abca133-c3a3-47eb-98d1-acaa6743e025\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48fwzr"
Jul 07 11:36:39.899738 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:39.899593    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8abca133-c3a3-47eb-98d1-acaa6743e025-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48fwzr\" (UID: \"8abca133-c3a3-47eb-98d1-acaa6743e025\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48fwzr"
Jul 07 11:36:39.899738 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:39.899630    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8abca133-c3a3-47eb-98d1-acaa6743e025-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48fwzr\" (UID: \"8abca133-c3a3-47eb-98d1-acaa6743e025\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48fwzr"
Jul 07 11:36:39.899738 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:39.899695    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dl2sl\" (UniqueName: \"kubernetes.io/projected/8abca133-c3a3-47eb-98d1-acaa6743e025-kube-api-access-dl2sl\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48fwzr\" (UID: \"8abca133-c3a3-47eb-98d1-acaa6743e025\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48fwzr"
Jul 07 11:36:39.899738 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:39.899731    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8abca133-c3a3-47eb-98d1-acaa6743e025-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48fwzr\" (UID: \"8abca133-c3a3-47eb-98d1-acaa6743e025\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48fwzr"
Jul 07 11:36:39.899938 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:39.899759    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8abca133-c3a3-47eb-98d1-acaa6743e025-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48fwzr\" (UID: \"8abca133-c3a3-47eb-98d1-acaa6743e025\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48fwzr"
Jul 07 11:36:39.899938 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:39.899802    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8abca133-c3a3-47eb-98d1-acaa6743e025-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48fwzr\" (UID: \"8abca133-c3a3-47eb-98d1-acaa6743e025\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48fwzr"
Jul 07 11:36:39.899938 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:39.899857    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8abca133-c3a3-47eb-98d1-acaa6743e025-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48fwzr\" (UID: \"8abca133-c3a3-47eb-98d1-acaa6743e025\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48fwzr"
Jul 07 11:36:39.899938 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:39.899884    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8abca133-c3a3-47eb-98d1-acaa6743e025-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48fwzr\" (UID: \"8abca133-c3a3-47eb-98d1-acaa6743e025\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48fwzr"
Jul 07 11:36:40.000589 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:40.000536    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8abca133-c3a3-47eb-98d1-acaa6743e025-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48fwzr\" (UID: \"8abca133-c3a3-47eb-98d1-acaa6743e025\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48fwzr"
Jul 07 11:36:40.000778 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:40.000606    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8abca133-c3a3-47eb-98d1-acaa6743e025-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48fwzr\" (UID: \"8abca133-c3a3-47eb-98d1-acaa6743e025\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48fwzr"
Jul 07 11:36:40.000778 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:40.000632    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8abca133-c3a3-47eb-98d1-acaa6743e025-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48fwzr\" (UID: \"8abca133-c3a3-47eb-98d1-acaa6743e025\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48fwzr"
Jul 07 11:36:40.000778 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:40.000654    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8abca133-c3a3-47eb-98d1-acaa6743e025-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48fwzr\" (UID: \"8abca133-c3a3-47eb-98d1-acaa6743e025\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48fwzr"
Jul 07 11:36:40.000778 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:40.000766    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8abca133-c3a3-47eb-98d1-acaa6743e025-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48fwzr\" (UID: \"8abca133-c3a3-47eb-98d1-acaa6743e025\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48fwzr"
Jul 07 11:36:40.000998 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:40.000829    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8abca133-c3a3-47eb-98d1-acaa6743e025-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48fwzr\" (UID: \"8abca133-c3a3-47eb-98d1-acaa6743e025\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48fwzr"
Jul 07 11:36:40.000998 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:40.000889    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dl2sl\" (UniqueName: \"kubernetes.io/projected/8abca133-c3a3-47eb-98d1-acaa6743e025-kube-api-access-dl2sl\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48fwzr\" (UID: \"8abca133-c3a3-47eb-98d1-acaa6743e025\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48fwzr"
Jul 07 11:36:40.000998 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:40.000929    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8abca133-c3a3-47eb-98d1-acaa6743e025-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48fwzr\" (UID: \"8abca133-c3a3-47eb-98d1-acaa6743e025\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48fwzr"
Jul 07 11:36:40.000998 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:40.000978    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8abca133-c3a3-47eb-98d1-acaa6743e025-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48fwzr\" (UID: \"8abca133-c3a3-47eb-98d1-acaa6743e025\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48fwzr"
Jul 07 11:36:40.001202 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:40.001082    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8abca133-c3a3-47eb-98d1-acaa6743e025-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48fwzr\" (UID: \"8abca133-c3a3-47eb-98d1-acaa6743e025\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48fwzr"
Jul 07 11:36:40.001202 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:40.001140    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8abca133-c3a3-47eb-98d1-acaa6743e025-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48fwzr\" (UID: \"8abca133-c3a3-47eb-98d1-acaa6743e025\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48fwzr"
Jul 07 11:36:40.001434 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:40.001398    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8abca133-c3a3-47eb-98d1-acaa6743e025-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48fwzr\" (UID: \"8abca133-c3a3-47eb-98d1-acaa6743e025\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48fwzr"
Jul 07 11:36:40.001561 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:40.001460    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8abca133-c3a3-47eb-98d1-acaa6743e025-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48fwzr\" (UID: \"8abca133-c3a3-47eb-98d1-acaa6743e025\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48fwzr"
Jul 07 11:36:40.001678 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:40.001658    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8abca133-c3a3-47eb-98d1-acaa6743e025-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48fwzr\" (UID: \"8abca133-c3a3-47eb-98d1-acaa6743e025\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48fwzr"
Jul 07 11:36:40.002894 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:40.002872    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8abca133-c3a3-47eb-98d1-acaa6743e025-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48fwzr\" (UID: \"8abca133-c3a3-47eb-98d1-acaa6743e025\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48fwzr"
Jul 07 11:36:40.003285 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:40.003264    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8abca133-c3a3-47eb-98d1-acaa6743e025-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48fwzr\" (UID: \"8abca133-c3a3-47eb-98d1-acaa6743e025\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48fwzr"
Jul 07 11:36:40.016366 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:40.016342    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8abca133-c3a3-47eb-98d1-acaa6743e025-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48fwzr\" (UID: \"8abca133-c3a3-47eb-98d1-acaa6743e025\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48fwzr"
Jul 07 11:36:40.016466 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:40.016449    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dl2sl\" (UniqueName: \"kubernetes.io/projected/8abca133-c3a3-47eb-98d1-acaa6743e025-kube-api-access-dl2sl\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48fwzr\" (UID: \"8abca133-c3a3-47eb-98d1-acaa6743e025\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48fwzr"
Jul 07 11:36:40.126491 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:40.126403    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48fwzr"
Jul 07 11:36:40.334845 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:40.334766    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48fwzr"]
Jul 07 11:36:40.337550 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:36:40.337505    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8abca133_c3a3_47eb_98d1_acaa6743e025.slice/crio-4a1f558e42f93b44bd8063064b6acfe4dae18e064f6e2ab3c74a632e9c6e3455 WatchSource:0}: Error finding container 4a1f558e42f93b44bd8063064b6acfe4dae18e064f6e2ab3c74a632e9c6e3455: Status 404 returned error can't find the container with id 4a1f558e42f93b44bd8063064b6acfe4dae18e064f6e2ab3c74a632e9c6e3455
Jul 07 11:36:41.054358 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:41.054304    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48fwzr" event={"ID":"8abca133-c3a3-47eb-98d1-acaa6743e025","Type":"ContainerStarted","Data":"4a1f558e42f93b44bd8063064b6acfe4dae18e064f6e2ab3c74a632e9c6e3455"}
Jul 07 11:36:42.155329 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:42.155292    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-69986dd77b-dqqh9"]
Jul 07 11:36:42.162344 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:42.162322    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-69986dd77b-dqqh9"
Jul 07 11:36:42.167448 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:42.167393    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 07 11:36:42.167604 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:42.167396    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 07 11:36:42.168831 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:42.168808    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-m9542\""
Jul 07 11:36:42.168997 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:42.168857    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 07 11:36:42.168997 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:42.168968    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 07 11:36:42.173959 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:42.173936    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 07 11:36:42.179709 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:42.179595    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-69986dd77b-dqqh9"]
Jul 07 11:36:42.321689 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:42.321645    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/44444391-e155-47ac-b321-d421f0c55a52-metrics-cert\") pod \"lws-controller-manager-69986dd77b-dqqh9\" (UID: \"44444391-e155-47ac-b321-d421f0c55a52\") " pod="openshift-lws-operator/lws-controller-manager-69986dd77b-dqqh9"
Jul 07 11:36:42.321689 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:42.321690    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4sgtl\" (UniqueName: \"kubernetes.io/projected/44444391-e155-47ac-b321-d421f0c55a52-kube-api-access-4sgtl\") pod \"lws-controller-manager-69986dd77b-dqqh9\" (UID: \"44444391-e155-47ac-b321-d421f0c55a52\") " pod="openshift-lws-operator/lws-controller-manager-69986dd77b-dqqh9"
Jul 07 11:36:42.321990 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:42.321795    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/44444391-e155-47ac-b321-d421f0c55a52-manager-config\") pod \"lws-controller-manager-69986dd77b-dqqh9\" (UID: \"44444391-e155-47ac-b321-d421f0c55a52\") " pod="openshift-lws-operator/lws-controller-manager-69986dd77b-dqqh9"
Jul 07 11:36:42.321990 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:42.321847    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/44444391-e155-47ac-b321-d421f0c55a52-cert\") pod \"lws-controller-manager-69986dd77b-dqqh9\" (UID: \"44444391-e155-47ac-b321-d421f0c55a52\") " pod="openshift-lws-operator/lws-controller-manager-69986dd77b-dqqh9"
Jul 07 11:36:42.423048 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:42.422954    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/44444391-e155-47ac-b321-d421f0c55a52-metrics-cert\") pod \"lws-controller-manager-69986dd77b-dqqh9\" (UID: \"44444391-e155-47ac-b321-d421f0c55a52\") " pod="openshift-lws-operator/lws-controller-manager-69986dd77b-dqqh9"
Jul 07 11:36:42.423048 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:42.423004    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4sgtl\" (UniqueName: \"kubernetes.io/projected/44444391-e155-47ac-b321-d421f0c55a52-kube-api-access-4sgtl\") pod \"lws-controller-manager-69986dd77b-dqqh9\" (UID: \"44444391-e155-47ac-b321-d421f0c55a52\") " pod="openshift-lws-operator/lws-controller-manager-69986dd77b-dqqh9"
Jul 07 11:36:42.423048 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:42.423050    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/44444391-e155-47ac-b321-d421f0c55a52-manager-config\") pod \"lws-controller-manager-69986dd77b-dqqh9\" (UID: \"44444391-e155-47ac-b321-d421f0c55a52\") " pod="openshift-lws-operator/lws-controller-manager-69986dd77b-dqqh9"
Jul 07 11:36:42.423380 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:42.423076    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/44444391-e155-47ac-b321-d421f0c55a52-cert\") pod \"lws-controller-manager-69986dd77b-dqqh9\" (UID: \"44444391-e155-47ac-b321-d421f0c55a52\") " pod="openshift-lws-operator/lws-controller-manager-69986dd77b-dqqh9"
Jul 07 11:36:42.423909 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:42.423884    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/44444391-e155-47ac-b321-d421f0c55a52-manager-config\") pod \"lws-controller-manager-69986dd77b-dqqh9\" (UID: \"44444391-e155-47ac-b321-d421f0c55a52\") " pod="openshift-lws-operator/lws-controller-manager-69986dd77b-dqqh9"
Jul 07 11:36:42.425776 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:42.425752    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/44444391-e155-47ac-b321-d421f0c55a52-metrics-cert\") pod \"lws-controller-manager-69986dd77b-dqqh9\" (UID: \"44444391-e155-47ac-b321-d421f0c55a52\") " pod="openshift-lws-operator/lws-controller-manager-69986dd77b-dqqh9"
Jul 07 11:36:42.425883 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:42.425785    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/44444391-e155-47ac-b321-d421f0c55a52-cert\") pod \"lws-controller-manager-69986dd77b-dqqh9\" (UID: \"44444391-e155-47ac-b321-d421f0c55a52\") " pod="openshift-lws-operator/lws-controller-manager-69986dd77b-dqqh9"
Jul 07 11:36:42.433930 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:42.433903    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4sgtl\" (UniqueName: \"kubernetes.io/projected/44444391-e155-47ac-b321-d421f0c55a52-kube-api-access-4sgtl\") pod \"lws-controller-manager-69986dd77b-dqqh9\" (UID: \"44444391-e155-47ac-b321-d421f0c55a52\") " pod="openshift-lws-operator/lws-controller-manager-69986dd77b-dqqh9"
Jul 07 11:36:42.474907 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:42.474877    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-69986dd77b-dqqh9"
Jul 07 11:36:43.019356 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:43.017290    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 11:36:43.019356 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:43.017383    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 11:36:43.019356 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:43.017426    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 11:36:43.144718 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:43.144588    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-69986dd77b-dqqh9"]
Jul 07 11:36:43.146961 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:36:43.146928    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44444391_e155_47ac_b321_d421f0c55a52.slice/crio-fe9726e9675fc78c4b199c527b3e78aeed2cceb5b60219c424fea8447e3171fd WatchSource:0}: Error finding container fe9726e9675fc78c4b199c527b3e78aeed2cceb5b60219c424fea8447e3171fd: Status 404 returned error can't find the container with id fe9726e9675fc78c4b199c527b3e78aeed2cceb5b60219c424fea8447e3171fd
Jul 07 11:36:44.074898 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:44.074860    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-69986dd77b-dqqh9" event={"ID":"44444391-e155-47ac-b321-d421f0c55a52","Type":"ContainerStarted","Data":"fe9726e9675fc78c4b199c527b3e78aeed2cceb5b60219c424fea8447e3171fd"}
Jul 07 11:36:44.076361 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:44.076334    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48fwzr" event={"ID":"8abca133-c3a3-47eb-98d1-acaa6743e025","Type":"ContainerStarted","Data":"59bda655df90507230a866f071c59a5747891996fd53d2e84bd548b67b091ae8"}
Jul 07 11:36:44.127245 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:44.127191    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48fwzr"
Jul 07 11:36:44.131866 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:44.131837    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48fwzr"
Jul 07 11:36:44.154032 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:44.153972    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48fwzr" podStartSLOduration=2.476791867 podStartE2EDuration="5.153956391s" totalImagesPullingTime="2.677164524s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:36:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:36:44.10215147 +0000 UTC m=+592.075791881" watchObservedRunningTime="2026-07-07 11:36:44.153956391 +0000 UTC m=+592.127596801"
Jul 07 11:36:45.081362 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:45.081329    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48fwzr"
Jul 07 11:36:45.082329 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:45.082312    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48fwzr"
Jul 07 11:36:49.102004 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:49.101925    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-69986dd77b-dqqh9" event={"ID":"44444391-e155-47ac-b321-d421f0c55a52","Type":"ContainerStarted","Data":"859f2ef434ce98cfd6174ebe8dc1d184bd8374e178e9a796d60144d4c57bc962"}
Jul 07 11:36:49.102497 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:49.102047    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-69986dd77b-dqqh9"
Jul 07 11:36:49.121041 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:49.120992    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-69986dd77b-dqqh9" podStartSLOduration=1.5592039 podStartE2EDuration="7.120977309s" totalImagesPullingTime="5.561773409s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:36:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:36:49.119368145 +0000 UTC m=+597.093008555" watchObservedRunningTime="2026-07-07 11:36:49.120977309 +0000 UTC m=+597.094617783"
Jul 07 11:36:50.050838 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:50.050810    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-7dl7x"
Jul 07 11:36:53.860297 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:53.860269    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ktbpw_faf6f343-6515-4b2c-a34e-286cd2d9ec7a/console-operator/1.log"
Jul 07 11:36:53.860763 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:36:53.860740    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ktbpw_faf6f343-6515-4b2c-a34e-286cd2d9ec7a/console-operator/1.log"
Jul 07 11:37:00.108482 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:00.108448    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-69986dd77b-dqqh9"
Jul 07 11:37:47.980231 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:47.980180    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h75kt"]
Jul 07 11:37:47.983845 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:47.983821    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h75kt"
Jul 07 11:37:47.986742 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:47.986715    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 07 11:37:47.986863 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:47.986747    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 07 11:37:47.986863 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:47.986834    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-2cjpz\""
Jul 07 11:37:47.992394 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:47.992363    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h75kt"]
Jul 07 11:37:48.075758 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:48.075726    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bh8g2\" (UniqueName: \"kubernetes.io/projected/536b501c-5d9f-4b9b-9bcd-296787d774c7-kube-api-access-bh8g2\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h75kt\" (UID: \"536b501c-5d9f-4b9b-9bcd-296787d774c7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h75kt"
Jul 07 11:37:48.075920 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:48.075771    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/536b501c-5d9f-4b9b-9bcd-296787d774c7-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h75kt\" (UID: \"536b501c-5d9f-4b9b-9bcd-296787d774c7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h75kt"
Jul 07 11:37:48.075920 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:48.075825    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/536b501c-5d9f-4b9b-9bcd-296787d774c7-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h75kt\" (UID: \"536b501c-5d9f-4b9b-9bcd-296787d774c7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h75kt"
Jul 07 11:37:48.176369 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:48.176328    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/536b501c-5d9f-4b9b-9bcd-296787d774c7-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h75kt\" (UID: \"536b501c-5d9f-4b9b-9bcd-296787d774c7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h75kt"
Jul 07 11:37:48.176562 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:48.176422    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bh8g2\" (UniqueName: \"kubernetes.io/projected/536b501c-5d9f-4b9b-9bcd-296787d774c7-kube-api-access-bh8g2\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h75kt\" (UID: \"536b501c-5d9f-4b9b-9bcd-296787d774c7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h75kt"
Jul 07 11:37:48.176562 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:48.176451    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/536b501c-5d9f-4b9b-9bcd-296787d774c7-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h75kt\" (UID: \"536b501c-5d9f-4b9b-9bcd-296787d774c7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h75kt"
Jul 07 11:37:48.176735 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:48.176714    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/536b501c-5d9f-4b9b-9bcd-296787d774c7-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h75kt\" (UID: \"536b501c-5d9f-4b9b-9bcd-296787d774c7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h75kt"
Jul 07 11:37:48.176797 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:48.176743    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/536b501c-5d9f-4b9b-9bcd-296787d774c7-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h75kt\" (UID: \"536b501c-5d9f-4b9b-9bcd-296787d774c7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h75kt"
Jul 07 11:37:48.186418 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:48.186393    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bh8g2\" (UniqueName: \"kubernetes.io/projected/536b501c-5d9f-4b9b-9bcd-296787d774c7-kube-api-access-bh8g2\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h75kt\" (UID: \"536b501c-5d9f-4b9b-9bcd-296787d774c7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h75kt"
Jul 07 11:37:48.294914 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:48.294819    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h75kt"
Jul 07 11:37:48.434030 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:48.434003    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h75kt"]
Jul 07 11:37:48.436189 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:37:48.436162    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod536b501c_5d9f_4b9b_9bcd_296787d774c7.slice/crio-5b0a46145c71a7c1481c81cb3378c5092af5b75ca6a5cc40f19a5575745b29ba WatchSource:0}: Error finding container 5b0a46145c71a7c1481c81cb3378c5092af5b75ca6a5cc40f19a5575745b29ba: Status 404 returned error can't find the container with id 5b0a46145c71a7c1481c81cb3378c5092af5b75ca6a5cc40f19a5575745b29ba
Jul 07 11:37:48.793315 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:48.793282    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lns8w"]
Jul 07 11:37:48.796747 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:48.796725    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lns8w"
Jul 07 11:37:48.804877 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:48.804849    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lns8w"]
Jul 07 11:37:48.885987 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:48.885943    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4f247630-5817-4f35-8527-97024949c300-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lns8w\" (UID: \"4f247630-5817-4f35-8527-97024949c300\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lns8w"
Jul 07 11:37:48.886170 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:48.885999    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4f247630-5817-4f35-8527-97024949c300-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lns8w\" (UID: \"4f247630-5817-4f35-8527-97024949c300\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lns8w"
Jul 07 11:37:48.886170 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:48.886032    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ktvm\" (UniqueName: \"kubernetes.io/projected/4f247630-5817-4f35-8527-97024949c300-kube-api-access-9ktvm\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lns8w\" (UID: \"4f247630-5817-4f35-8527-97024949c300\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lns8w"
Jul 07 11:37:48.987319 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:48.987275    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4f247630-5817-4f35-8527-97024949c300-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lns8w\" (UID: \"4f247630-5817-4f35-8527-97024949c300\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lns8w"
Jul 07 11:37:48.987704 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:48.987328    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4f247630-5817-4f35-8527-97024949c300-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lns8w\" (UID: \"4f247630-5817-4f35-8527-97024949c300\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lns8w"
Jul 07 11:37:48.987704 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:48.987356    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ktvm\" (UniqueName: \"kubernetes.io/projected/4f247630-5817-4f35-8527-97024949c300-kube-api-access-9ktvm\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lns8w\" (UID: \"4f247630-5817-4f35-8527-97024949c300\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lns8w"
Jul 07 11:37:48.987704 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:48.987669    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4f247630-5817-4f35-8527-97024949c300-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lns8w\" (UID: \"4f247630-5817-4f35-8527-97024949c300\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lns8w"
Jul 07 11:37:48.987704 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:48.987696    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4f247630-5817-4f35-8527-97024949c300-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lns8w\" (UID: \"4f247630-5817-4f35-8527-97024949c300\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lns8w"
Jul 07 11:37:48.996132 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:48.996107    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ktvm\" (UniqueName: \"kubernetes.io/projected/4f247630-5817-4f35-8527-97024949c300-kube-api-access-9ktvm\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lns8w\" (UID: \"4f247630-5817-4f35-8527-97024949c300\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lns8w"
Jul 07 11:37:49.107310 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:49.107226    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lns8w"
Jul 07 11:37:49.247003 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:49.246860    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lns8w"]
Jul 07 11:37:49.249169 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:37:49.249137    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f247630_5817_4f35_8527_97024949c300.slice/crio-26e15c5aaca613d123021e2a3c86af030901556ceb72373cb7cc3330dc28213e WatchSource:0}: Error finding container 26e15c5aaca613d123021e2a3c86af030901556ceb72373cb7cc3330dc28213e: Status 404 returned error can't find the container with id 26e15c5aaca613d123021e2a3c86af030901556ceb72373cb7cc3330dc28213e
Jul 07 11:37:49.373486 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:49.373454    2570 generic.go:356] "Generic (PLEG): container finished" podID="4f247630-5817-4f35-8527-97024949c300" containerID="a7247621bf056d277062c7e0ac174377083bd72bc596f7795ba46068f54917fc" exitCode=0
Jul 07 11:37:49.373632 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:49.373540    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lns8w" event={"ID":"4f247630-5817-4f35-8527-97024949c300","Type":"ContainerDied","Data":"a7247621bf056d277062c7e0ac174377083bd72bc596f7795ba46068f54917fc"}
Jul 07 11:37:49.373632 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:49.373612    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lns8w" event={"ID":"4f247630-5817-4f35-8527-97024949c300","Type":"ContainerStarted","Data":"26e15c5aaca613d123021e2a3c86af030901556ceb72373cb7cc3330dc28213e"}
Jul 07 11:37:49.374091 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:49.374067    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 11:37:49.375268 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:49.375244    2570 generic.go:356] "Generic (PLEG): container finished" podID="536b501c-5d9f-4b9b-9bcd-296787d774c7" containerID="67e8e391ac52be9587b9f5be71db94b0f68d10ed932faad9218d921d3616b994" exitCode=0
Jul 07 11:37:49.375362 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:49.375285    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h75kt" event={"ID":"536b501c-5d9f-4b9b-9bcd-296787d774c7","Type":"ContainerDied","Data":"67e8e391ac52be9587b9f5be71db94b0f68d10ed932faad9218d921d3616b994"}
Jul 07 11:37:49.375362 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:49.375314    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h75kt" event={"ID":"536b501c-5d9f-4b9b-9bcd-296787d774c7","Type":"ContainerStarted","Data":"5b0a46145c71a7c1481c81cb3378c5092af5b75ca6a5cc40f19a5575745b29ba"}
Jul 07 11:37:49.386365 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:49.386339    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cpcql"]
Jul 07 11:37:49.391156 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:49.391138    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cpcql"
Jul 07 11:37:49.398154 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:49.398130    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cpcql"]
Jul 07 11:37:49.591429 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:49.591388    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fce422a1-973a-4cd1-a894-a0f2e7706f8f-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cpcql\" (UID: \"fce422a1-973a-4cd1-a894-a0f2e7706f8f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cpcql"
Jul 07 11:37:49.591429 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:49.591429    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsvk6\" (UniqueName: \"kubernetes.io/projected/fce422a1-973a-4cd1-a894-a0f2e7706f8f-kube-api-access-hsvk6\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cpcql\" (UID: \"fce422a1-973a-4cd1-a894-a0f2e7706f8f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cpcql"
Jul 07 11:37:49.591633 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:49.591521    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fce422a1-973a-4cd1-a894-a0f2e7706f8f-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cpcql\" (UID: \"fce422a1-973a-4cd1-a894-a0f2e7706f8f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cpcql"
Jul 07 11:37:49.691981 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:49.691951    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fce422a1-973a-4cd1-a894-a0f2e7706f8f-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cpcql\" (UID: \"fce422a1-973a-4cd1-a894-a0f2e7706f8f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cpcql"
Jul 07 11:37:49.692141 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:49.692018    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fce422a1-973a-4cd1-a894-a0f2e7706f8f-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cpcql\" (UID: \"fce422a1-973a-4cd1-a894-a0f2e7706f8f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cpcql"
Jul 07 11:37:49.692141 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:49.692054    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsvk6\" (UniqueName: \"kubernetes.io/projected/fce422a1-973a-4cd1-a894-a0f2e7706f8f-kube-api-access-hsvk6\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cpcql\" (UID: \"fce422a1-973a-4cd1-a894-a0f2e7706f8f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cpcql"
Jul 07 11:37:49.692380 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:49.692360    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fce422a1-973a-4cd1-a894-a0f2e7706f8f-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cpcql\" (UID: \"fce422a1-973a-4cd1-a894-a0f2e7706f8f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cpcql"
Jul 07 11:37:49.692425 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:49.692405    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fce422a1-973a-4cd1-a894-a0f2e7706f8f-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cpcql\" (UID: \"fce422a1-973a-4cd1-a894-a0f2e7706f8f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cpcql"
Jul 07 11:37:49.701319 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:49.701290    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsvk6\" (UniqueName: \"kubernetes.io/projected/fce422a1-973a-4cd1-a894-a0f2e7706f8f-kube-api-access-hsvk6\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cpcql\" (UID: \"fce422a1-973a-4cd1-a894-a0f2e7706f8f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cpcql"
Jul 07 11:37:49.720108 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:49.720079    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cpcql"
Jul 07 11:37:49.783000 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:49.782962    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nd2r4"]
Jul 07 11:37:49.785674 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:49.785651    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nd2r4"
Jul 07 11:37:49.793050 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:49.792979    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79trk\" (UniqueName: \"kubernetes.io/projected/4d631fc5-7f71-4c37-bcd3-59a88dde0305-kube-api-access-79trk\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nd2r4\" (UID: \"4d631fc5-7f71-4c37-bcd3-59a88dde0305\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nd2r4"
Jul 07 11:37:49.793050 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:49.793030    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4d631fc5-7f71-4c37-bcd3-59a88dde0305-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nd2r4\" (UID: \"4d631fc5-7f71-4c37-bcd3-59a88dde0305\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nd2r4"
Jul 07 11:37:49.793823 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:49.793117    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4d631fc5-7f71-4c37-bcd3-59a88dde0305-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nd2r4\" (UID: \"4d631fc5-7f71-4c37-bcd3-59a88dde0305\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nd2r4"
Jul 07 11:37:49.795390 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:49.795363    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nd2r4"]
Jul 07 11:37:49.861822 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:49.858900    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cpcql"]
Jul 07 11:37:49.862461 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:37:49.862434    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfce422a1_973a_4cd1_a894_a0f2e7706f8f.slice/crio-02ecf952149bdfd7593511ba284f5f08339de4860b369e57efa2df7a316c666a WatchSource:0}: Error finding container 02ecf952149bdfd7593511ba284f5f08339de4860b369e57efa2df7a316c666a: Status 404 returned error can't find the container with id 02ecf952149bdfd7593511ba284f5f08339de4860b369e57efa2df7a316c666a
Jul 07 11:37:49.893889 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:49.893859    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79trk\" (UniqueName: \"kubernetes.io/projected/4d631fc5-7f71-4c37-bcd3-59a88dde0305-kube-api-access-79trk\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nd2r4\" (UID: \"4d631fc5-7f71-4c37-bcd3-59a88dde0305\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nd2r4"
Jul 07 11:37:49.894044 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:49.893896    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4d631fc5-7f71-4c37-bcd3-59a88dde0305-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nd2r4\" (UID: \"4d631fc5-7f71-4c37-bcd3-59a88dde0305\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nd2r4"
Jul 07 11:37:49.894044 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:49.893951    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4d631fc5-7f71-4c37-bcd3-59a88dde0305-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nd2r4\" (UID: \"4d631fc5-7f71-4c37-bcd3-59a88dde0305\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nd2r4"
Jul 07 11:37:49.894380 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:49.894359    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4d631fc5-7f71-4c37-bcd3-59a88dde0305-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nd2r4\" (UID: \"4d631fc5-7f71-4c37-bcd3-59a88dde0305\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nd2r4"
Jul 07 11:37:49.894380 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:49.894372    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4d631fc5-7f71-4c37-bcd3-59a88dde0305-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nd2r4\" (UID: \"4d631fc5-7f71-4c37-bcd3-59a88dde0305\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nd2r4"
Jul 07 11:37:49.902977 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:49.902952    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-79trk\" (UniqueName: \"kubernetes.io/projected/4d631fc5-7f71-4c37-bcd3-59a88dde0305-kube-api-access-79trk\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nd2r4\" (UID: \"4d631fc5-7f71-4c37-bcd3-59a88dde0305\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nd2r4"
Jul 07 11:37:50.096279 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:50.096243    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nd2r4"
Jul 07 11:37:50.384773 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:50.384684    2570 generic.go:356] "Generic (PLEG): container finished" podID="fce422a1-973a-4cd1-a894-a0f2e7706f8f" containerID="d4292421f1f69d7f481e2eb2faead3ac485ef9ede502ef116e3ae67bf4680c29" exitCode=0
Jul 07 11:37:50.384904 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:50.384771    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cpcql" event={"ID":"fce422a1-973a-4cd1-a894-a0f2e7706f8f","Type":"ContainerDied","Data":"d4292421f1f69d7f481e2eb2faead3ac485ef9ede502ef116e3ae67bf4680c29"}
Jul 07 11:37:50.384904 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:50.384816    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cpcql" event={"ID":"fce422a1-973a-4cd1-a894-a0f2e7706f8f","Type":"ContainerStarted","Data":"02ecf952149bdfd7593511ba284f5f08339de4860b369e57efa2df7a316c666a"}
Jul 07 11:37:50.425351 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:50.425280    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nd2r4"]
Jul 07 11:37:50.426968 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:37:50.426932    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d631fc5_7f71_4c37_bcd3_59a88dde0305.slice/crio-8689357086f3dab070c5bb280adbaccaab7443d5129f5ec09dbdf473a03573c7 WatchSource:0}: Error finding container 8689357086f3dab070c5bb280adbaccaab7443d5129f5ec09dbdf473a03573c7: Status 404 returned error can't find the container with id 8689357086f3dab070c5bb280adbaccaab7443d5129f5ec09dbdf473a03573c7
Jul 07 11:37:51.394042 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:51.393944    2570 generic.go:356] "Generic (PLEG): container finished" podID="536b501c-5d9f-4b9b-9bcd-296787d774c7" containerID="3393bdb3f777f94597db1acaa5bad32798b16d1825833d2fcbb9d870836124a6" exitCode=0
Jul 07 11:37:51.394042 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:51.394027    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h75kt" event={"ID":"536b501c-5d9f-4b9b-9bcd-296787d774c7","Type":"ContainerDied","Data":"3393bdb3f777f94597db1acaa5bad32798b16d1825833d2fcbb9d870836124a6"}
Jul 07 11:37:51.396159 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:51.396133    2570 generic.go:356] "Generic (PLEG): container finished" podID="fce422a1-973a-4cd1-a894-a0f2e7706f8f" containerID="ba5bba4a564b1c7c4b0052f0c725bd145f17e630584976677a8444edc3214887" exitCode=0
Jul 07 11:37:51.396260 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:51.396168    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cpcql" event={"ID":"fce422a1-973a-4cd1-a894-a0f2e7706f8f","Type":"ContainerDied","Data":"ba5bba4a564b1c7c4b0052f0c725bd145f17e630584976677a8444edc3214887"}
Jul 07 11:37:51.398256 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:51.398235    2570 generic.go:356] "Generic (PLEG): container finished" podID="4f247630-5817-4f35-8527-97024949c300" containerID="416e6e4a6533e3a0458fc6b5d8aafb5caec544229fe631afd8af3b89fe0da51b" exitCode=0
Jul 07 11:37:51.398342 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:51.398316    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lns8w" event={"ID":"4f247630-5817-4f35-8527-97024949c300","Type":"ContainerDied","Data":"416e6e4a6533e3a0458fc6b5d8aafb5caec544229fe631afd8af3b89fe0da51b"}
Jul 07 11:37:51.399987 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:51.399936    2570 generic.go:356] "Generic (PLEG): container finished" podID="4d631fc5-7f71-4c37-bcd3-59a88dde0305" containerID="dba93e5a1e256b282ef2b75ce98ebc3da1d4f182af2af6a1ed0790c735bf9c03" exitCode=0
Jul 07 11:37:51.399987 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:51.399972    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nd2r4" event={"ID":"4d631fc5-7f71-4c37-bcd3-59a88dde0305","Type":"ContainerDied","Data":"dba93e5a1e256b282ef2b75ce98ebc3da1d4f182af2af6a1ed0790c735bf9c03"}
Jul 07 11:37:51.400116 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:51.400011    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nd2r4" event={"ID":"4d631fc5-7f71-4c37-bcd3-59a88dde0305","Type":"ContainerStarted","Data":"8689357086f3dab070c5bb280adbaccaab7443d5129f5ec09dbdf473a03573c7"}
Jul 07 11:37:52.406635 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:52.406601    2570 generic.go:356] "Generic (PLEG): container finished" podID="4d631fc5-7f71-4c37-bcd3-59a88dde0305" containerID="6e6b36bdb2791b86c561295ed733915d44335c9d992cdb9cf81ed2f4f5e30847" exitCode=0
Jul 07 11:37:52.407114 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:52.406663    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nd2r4" event={"ID":"4d631fc5-7f71-4c37-bcd3-59a88dde0305","Type":"ContainerDied","Data":"6e6b36bdb2791b86c561295ed733915d44335c9d992cdb9cf81ed2f4f5e30847"}
Jul 07 11:37:52.408872 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:52.408850    2570 generic.go:356] "Generic (PLEG): container finished" podID="536b501c-5d9f-4b9b-9bcd-296787d774c7" containerID="adfc504076cade502c608873990bee5436441e01180cffacd1cca42acac09b37" exitCode=0
Jul 07 11:37:52.408985 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:52.408938    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h75kt" event={"ID":"536b501c-5d9f-4b9b-9bcd-296787d774c7","Type":"ContainerDied","Data":"adfc504076cade502c608873990bee5436441e01180cffacd1cca42acac09b37"}
Jul 07 11:37:52.410994 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:52.410969    2570 generic.go:356] "Generic (PLEG): container finished" podID="fce422a1-973a-4cd1-a894-a0f2e7706f8f" containerID="2742b119a3d112fcd4818fdd495cd718a314d930df882a13b7d1a2ed119e3b77" exitCode=0
Jul 07 11:37:52.411077 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:52.411048    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cpcql" event={"ID":"fce422a1-973a-4cd1-a894-a0f2e7706f8f","Type":"ContainerDied","Data":"2742b119a3d112fcd4818fdd495cd718a314d930df882a13b7d1a2ed119e3b77"}
Jul 07 11:37:52.413247 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:52.413182    2570 generic.go:356] "Generic (PLEG): container finished" podID="4f247630-5817-4f35-8527-97024949c300" containerID="03ed04d9adf2a7dfad335742cce91237aff676b987ec674e96fa6d3fd778c087" exitCode=0
Jul 07 11:37:52.413247 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:52.413240    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lns8w" event={"ID":"4f247630-5817-4f35-8527-97024949c300","Type":"ContainerDied","Data":"03ed04d9adf2a7dfad335742cce91237aff676b987ec674e96fa6d3fd778c087"}
Jul 07 11:37:53.420054 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:53.420014    2570 generic.go:356] "Generic (PLEG): container finished" podID="4d631fc5-7f71-4c37-bcd3-59a88dde0305" containerID="ff039657cacd7dfef53606ff5b911ab95dca6765df94a55de57e455706d3c71a" exitCode=0
Jul 07 11:37:53.420654 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:53.420093    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nd2r4" event={"ID":"4d631fc5-7f71-4c37-bcd3-59a88dde0305","Type":"ContainerDied","Data":"ff039657cacd7dfef53606ff5b911ab95dca6765df94a55de57e455706d3c71a"}
Jul 07 11:37:53.563797 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:53.563761    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cpcql"
Jul 07 11:37:53.613642 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:53.613618    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lns8w"
Jul 07 11:37:53.616721 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:53.616698    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h75kt"
Jul 07 11:37:53.623830 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:53.623811    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/536b501c-5d9f-4b9b-9bcd-296787d774c7-util\") pod \"536b501c-5d9f-4b9b-9bcd-296787d774c7\" (UID: \"536b501c-5d9f-4b9b-9bcd-296787d774c7\") "
Jul 07 11:37:53.623947 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:53.623851    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/536b501c-5d9f-4b9b-9bcd-296787d774c7-bundle\") pod \"536b501c-5d9f-4b9b-9bcd-296787d774c7\" (UID: \"536b501c-5d9f-4b9b-9bcd-296787d774c7\") "
Jul 07 11:37:53.623947 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:53.623869    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fce422a1-973a-4cd1-a894-a0f2e7706f8f-bundle\") pod \"fce422a1-973a-4cd1-a894-a0f2e7706f8f\" (UID: \"fce422a1-973a-4cd1-a894-a0f2e7706f8f\") "
Jul 07 11:37:53.623947 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:53.623886    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fce422a1-973a-4cd1-a894-a0f2e7706f8f-util\") pod \"fce422a1-973a-4cd1-a894-a0f2e7706f8f\" (UID: \"fce422a1-973a-4cd1-a894-a0f2e7706f8f\") "
Jul 07 11:37:53.623947 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:53.623910    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9ktvm\" (UniqueName: \"kubernetes.io/projected/4f247630-5817-4f35-8527-97024949c300-kube-api-access-9ktvm\") pod \"4f247630-5817-4f35-8527-97024949c300\" (UID: \"4f247630-5817-4f35-8527-97024949c300\") "
Jul 07 11:37:53.623947 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:53.623943    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4f247630-5817-4f35-8527-97024949c300-util\") pod \"4f247630-5817-4f35-8527-97024949c300\" (UID: \"4f247630-5817-4f35-8527-97024949c300\") "
Jul 07 11:37:53.624265 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:53.623983    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hsvk6\" (UniqueName: \"kubernetes.io/projected/fce422a1-973a-4cd1-a894-a0f2e7706f8f-kube-api-access-hsvk6\") pod \"fce422a1-973a-4cd1-a894-a0f2e7706f8f\" (UID: \"fce422a1-973a-4cd1-a894-a0f2e7706f8f\") "
Jul 07 11:37:53.624265 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:53.624019    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bh8g2\" (UniqueName: \"kubernetes.io/projected/536b501c-5d9f-4b9b-9bcd-296787d774c7-kube-api-access-bh8g2\") pod \"536b501c-5d9f-4b9b-9bcd-296787d774c7\" (UID: \"536b501c-5d9f-4b9b-9bcd-296787d774c7\") "
Jul 07 11:37:53.624265 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:53.624050    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4f247630-5817-4f35-8527-97024949c300-bundle\") pod \"4f247630-5817-4f35-8527-97024949c300\" (UID: \"4f247630-5817-4f35-8527-97024949c300\") "
Jul 07 11:37:53.624439 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:53.624352    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/536b501c-5d9f-4b9b-9bcd-296787d774c7-bundle" (OuterVolumeSpecName: "bundle") pod "536b501c-5d9f-4b9b-9bcd-296787d774c7" (UID: "536b501c-5d9f-4b9b-9bcd-296787d774c7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:37:53.624439 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:53.624412    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fce422a1-973a-4cd1-a894-a0f2e7706f8f-bundle" (OuterVolumeSpecName: "bundle") pod "fce422a1-973a-4cd1-a894-a0f2e7706f8f" (UID: "fce422a1-973a-4cd1-a894-a0f2e7706f8f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:37:53.625167 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:53.625143    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f247630-5817-4f35-8527-97024949c300-bundle" (OuterVolumeSpecName: "bundle") pod "4f247630-5817-4f35-8527-97024949c300" (UID: "4f247630-5817-4f35-8527-97024949c300"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:37:53.626741 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:53.626715    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f247630-5817-4f35-8527-97024949c300-kube-api-access-9ktvm" (OuterVolumeSpecName: "kube-api-access-9ktvm") pod "4f247630-5817-4f35-8527-97024949c300" (UID: "4f247630-5817-4f35-8527-97024949c300"). InnerVolumeSpecName "kube-api-access-9ktvm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:37:53.626741 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:53.626734    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fce422a1-973a-4cd1-a894-a0f2e7706f8f-kube-api-access-hsvk6" (OuterVolumeSpecName: "kube-api-access-hsvk6") pod "fce422a1-973a-4cd1-a894-a0f2e7706f8f" (UID: "fce422a1-973a-4cd1-a894-a0f2e7706f8f"). InnerVolumeSpecName "kube-api-access-hsvk6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:37:53.627002 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:53.626973    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/536b501c-5d9f-4b9b-9bcd-296787d774c7-kube-api-access-bh8g2" (OuterVolumeSpecName: "kube-api-access-bh8g2") pod "536b501c-5d9f-4b9b-9bcd-296787d774c7" (UID: "536b501c-5d9f-4b9b-9bcd-296787d774c7"). InnerVolumeSpecName "kube-api-access-bh8g2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:37:53.630788 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:53.630767    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f247630-5817-4f35-8527-97024949c300-util" (OuterVolumeSpecName: "util") pod "4f247630-5817-4f35-8527-97024949c300" (UID: "4f247630-5817-4f35-8527-97024949c300"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:37:53.632169 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:53.632094    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fce422a1-973a-4cd1-a894-a0f2e7706f8f-util" (OuterVolumeSpecName: "util") pod "fce422a1-973a-4cd1-a894-a0f2e7706f8f" (UID: "fce422a1-973a-4cd1-a894-a0f2e7706f8f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:37:53.632811 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:53.632787    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/536b501c-5d9f-4b9b-9bcd-296787d774c7-util" (OuterVolumeSpecName: "util") pod "536b501c-5d9f-4b9b-9bcd-296787d774c7" (UID: "536b501c-5d9f-4b9b-9bcd-296787d774c7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:37:53.725201 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:53.725163    2570 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/536b501c-5d9f-4b9b-9bcd-296787d774c7-util\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:37:53.725201 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:53.725195    2570 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/536b501c-5d9f-4b9b-9bcd-296787d774c7-bundle\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:37:53.725201 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:53.725205    2570 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fce422a1-973a-4cd1-a894-a0f2e7706f8f-bundle\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:37:53.725462 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:53.725238    2570 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fce422a1-973a-4cd1-a894-a0f2e7706f8f-util\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:37:53.725462 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:53.725247    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9ktvm\" (UniqueName: \"kubernetes.io/projected/4f247630-5817-4f35-8527-97024949c300-kube-api-access-9ktvm\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:37:53.725462 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:53.725258    2570 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4f247630-5817-4f35-8527-97024949c300-util\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:37:53.725462 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:53.725266    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hsvk6\" (UniqueName: \"kubernetes.io/projected/fce422a1-973a-4cd1-a894-a0f2e7706f8f-kube-api-access-hsvk6\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:37:53.725462 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:53.725274    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bh8g2\" (UniqueName: \"kubernetes.io/projected/536b501c-5d9f-4b9b-9bcd-296787d774c7-kube-api-access-bh8g2\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:37:53.725462 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:53.725283    2570 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4f247630-5817-4f35-8527-97024949c300-bundle\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:37:54.427345 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:54.427316    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lns8w"
Jul 07 11:37:54.427771 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:54.427353    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lns8w" event={"ID":"4f247630-5817-4f35-8527-97024949c300","Type":"ContainerDied","Data":"26e15c5aaca613d123021e2a3c86af030901556ceb72373cb7cc3330dc28213e"}
Jul 07 11:37:54.427771 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:54.427400    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="26e15c5aaca613d123021e2a3c86af030901556ceb72373cb7cc3330dc28213e"
Jul 07 11:37:54.429410 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:54.429387    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h75kt"
Jul 07 11:37:54.429532 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:54.429449    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h75kt" event={"ID":"536b501c-5d9f-4b9b-9bcd-296787d774c7","Type":"ContainerDied","Data":"5b0a46145c71a7c1481c81cb3378c5092af5b75ca6a5cc40f19a5575745b29ba"}
Jul 07 11:37:54.429532 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:54.429499    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5b0a46145c71a7c1481c81cb3378c5092af5b75ca6a5cc40f19a5575745b29ba"
Jul 07 11:37:54.431766 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:54.431744    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cpcql" event={"ID":"fce422a1-973a-4cd1-a894-a0f2e7706f8f","Type":"ContainerDied","Data":"02ecf952149bdfd7593511ba284f5f08339de4860b369e57efa2df7a316c666a"}
Jul 07 11:37:54.431919 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:54.431782    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="02ecf952149bdfd7593511ba284f5f08339de4860b369e57efa2df7a316c666a"
Jul 07 11:37:54.431919 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:54.431907    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cpcql"
Jul 07 11:37:54.570975 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:54.570951    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nd2r4"
Jul 07 11:37:54.631476 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:54.631436    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-79trk\" (UniqueName: \"kubernetes.io/projected/4d631fc5-7f71-4c37-bcd3-59a88dde0305-kube-api-access-79trk\") pod \"4d631fc5-7f71-4c37-bcd3-59a88dde0305\" (UID: \"4d631fc5-7f71-4c37-bcd3-59a88dde0305\") "
Jul 07 11:37:54.631674 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:54.631512    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4d631fc5-7f71-4c37-bcd3-59a88dde0305-bundle\") pod \"4d631fc5-7f71-4c37-bcd3-59a88dde0305\" (UID: \"4d631fc5-7f71-4c37-bcd3-59a88dde0305\") "
Jul 07 11:37:54.631674 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:54.631574    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4d631fc5-7f71-4c37-bcd3-59a88dde0305-util\") pod \"4d631fc5-7f71-4c37-bcd3-59a88dde0305\" (UID: \"4d631fc5-7f71-4c37-bcd3-59a88dde0305\") "
Jul 07 11:37:54.632111 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:54.632071    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d631fc5-7f71-4c37-bcd3-59a88dde0305-bundle" (OuterVolumeSpecName: "bundle") pod "4d631fc5-7f71-4c37-bcd3-59a88dde0305" (UID: "4d631fc5-7f71-4c37-bcd3-59a88dde0305"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:37:54.633611 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:54.633583    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d631fc5-7f71-4c37-bcd3-59a88dde0305-kube-api-access-79trk" (OuterVolumeSpecName: "kube-api-access-79trk") pod "4d631fc5-7f71-4c37-bcd3-59a88dde0305" (UID: "4d631fc5-7f71-4c37-bcd3-59a88dde0305"). InnerVolumeSpecName "kube-api-access-79trk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:37:54.637104 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:54.637083    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d631fc5-7f71-4c37-bcd3-59a88dde0305-util" (OuterVolumeSpecName: "util") pod "4d631fc5-7f71-4c37-bcd3-59a88dde0305" (UID: "4d631fc5-7f71-4c37-bcd3-59a88dde0305"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:37:54.732573 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:54.732536    2570 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4d631fc5-7f71-4c37-bcd3-59a88dde0305-util\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:37:54.732573 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:54.732571    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-79trk\" (UniqueName: \"kubernetes.io/projected/4d631fc5-7f71-4c37-bcd3-59a88dde0305-kube-api-access-79trk\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:37:54.732791 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:54.732586    2570 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4d631fc5-7f71-4c37-bcd3-59a88dde0305-bundle\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:37:55.438082 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:55.438045    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nd2r4" event={"ID":"4d631fc5-7f71-4c37-bcd3-59a88dde0305","Type":"ContainerDied","Data":"8689357086f3dab070c5bb280adbaccaab7443d5129f5ec09dbdf473a03573c7"}
Jul 07 11:37:55.438082 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:55.438070    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nd2r4"
Jul 07 11:37:55.438557 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:37:55.438130    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8689357086f3dab070c5bb280adbaccaab7443d5129f5ec09dbdf473a03573c7"
Jul 07 11:39:12.028468 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.028434    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vjhwc"]
Jul 07 11:39:12.028944 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.028812    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="536b501c-5d9f-4b9b-9bcd-296787d774c7" containerName="util"
Jul 07 11:39:12.028944 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.028827    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="536b501c-5d9f-4b9b-9bcd-296787d774c7" containerName="util"
Jul 07 11:39:12.028944 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.028839    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f247630-5817-4f35-8527-97024949c300" containerName="util"
Jul 07 11:39:12.028944 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.028847    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f247630-5817-4f35-8527-97024949c300" containerName="util"
Jul 07 11:39:12.028944 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.028859    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d631fc5-7f71-4c37-bcd3-59a88dde0305" containerName="util"
Jul 07 11:39:12.028944 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.028865    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d631fc5-7f71-4c37-bcd3-59a88dde0305" containerName="util"
Jul 07 11:39:12.028944 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.028872    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fce422a1-973a-4cd1-a894-a0f2e7706f8f" containerName="pull"
Jul 07 11:39:12.028944 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.028877    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="fce422a1-973a-4cd1-a894-a0f2e7706f8f" containerName="pull"
Jul 07 11:39:12.028944 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.028884    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fce422a1-973a-4cd1-a894-a0f2e7706f8f" containerName="util"
Jul 07 11:39:12.028944 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.028890    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="fce422a1-973a-4cd1-a894-a0f2e7706f8f" containerName="util"
Jul 07 11:39:12.028944 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.028895    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f247630-5817-4f35-8527-97024949c300" containerName="pull"
Jul 07 11:39:12.028944 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.028900    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f247630-5817-4f35-8527-97024949c300" containerName="pull"
Jul 07 11:39:12.028944 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.028907    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d631fc5-7f71-4c37-bcd3-59a88dde0305" containerName="pull"
Jul 07 11:39:12.028944 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.028912    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d631fc5-7f71-4c37-bcd3-59a88dde0305" containerName="pull"
Jul 07 11:39:12.028944 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.028925    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d631fc5-7f71-4c37-bcd3-59a88dde0305" containerName="extract"
Jul 07 11:39:12.028944 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.028932    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d631fc5-7f71-4c37-bcd3-59a88dde0305" containerName="extract"
Jul 07 11:39:12.028944 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.028939    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="536b501c-5d9f-4b9b-9bcd-296787d774c7" containerName="extract"
Jul 07 11:39:12.028944 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.028944    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="536b501c-5d9f-4b9b-9bcd-296787d774c7" containerName="extract"
Jul 07 11:39:12.028944 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.028949    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fce422a1-973a-4cd1-a894-a0f2e7706f8f" containerName="extract"
Jul 07 11:39:12.028944 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.028954    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="fce422a1-973a-4cd1-a894-a0f2e7706f8f" containerName="extract"
Jul 07 11:39:12.029601 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.028962    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="536b501c-5d9f-4b9b-9bcd-296787d774c7" containerName="pull"
Jul 07 11:39:12.029601 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.028967    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="536b501c-5d9f-4b9b-9bcd-296787d774c7" containerName="pull"
Jul 07 11:39:12.029601 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.028974    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f247630-5817-4f35-8527-97024949c300" containerName="extract"
Jul 07 11:39:12.029601 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.028978    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f247630-5817-4f35-8527-97024949c300" containerName="extract"
Jul 07 11:39:12.029601 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.029034    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="536b501c-5d9f-4b9b-9bcd-296787d774c7" containerName="extract"
Jul 07 11:39:12.029601 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.029043    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f247630-5817-4f35-8527-97024949c300" containerName="extract"
Jul 07 11:39:12.029601 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.029049    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="fce422a1-973a-4cd1-a894-a0f2e7706f8f" containerName="extract"
Jul 07 11:39:12.029601 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.029055    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d631fc5-7f71-4c37-bcd3-59a88dde0305" containerName="extract"
Jul 07 11:39:12.032138 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.032119    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vjhwc"
Jul 07 11:39:12.034967 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.034941    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-cxsz9\""
Jul 07 11:39:12.043844 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.043820    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vjhwc"]
Jul 07 11:39:12.108175 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.108139    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b0afd6c-1e2a-4dee-b8fd-cde077988d1e-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-vjhwc\" (UID: \"5b0afd6c-1e2a-4dee-b8fd-cde077988d1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vjhwc"
Jul 07 11:39:12.108356 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.108182    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b0afd6c-1e2a-4dee-b8fd-cde077988d1e-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-vjhwc\" (UID: \"5b0afd6c-1e2a-4dee-b8fd-cde077988d1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vjhwc"
Jul 07 11:39:12.108356 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.108224    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b0afd6c-1e2a-4dee-b8fd-cde077988d1e-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-vjhwc\" (UID: \"5b0afd6c-1e2a-4dee-b8fd-cde077988d1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vjhwc"
Jul 07 11:39:12.108356 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.108309    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b0afd6c-1e2a-4dee-b8fd-cde077988d1e-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-vjhwc\" (UID: \"5b0afd6c-1e2a-4dee-b8fd-cde077988d1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vjhwc"
Jul 07 11:39:12.108356 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.108337    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b0afd6c-1e2a-4dee-b8fd-cde077988d1e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-vjhwc\" (UID: \"5b0afd6c-1e2a-4dee-b8fd-cde077988d1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vjhwc"
Jul 07 11:39:12.108513 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.108378    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdvs5\" (UniqueName: \"kubernetes.io/projected/5b0afd6c-1e2a-4dee-b8fd-cde077988d1e-kube-api-access-sdvs5\") pod \"maas-default-gateway-openshift-default-56654dcf7-vjhwc\" (UID: \"5b0afd6c-1e2a-4dee-b8fd-cde077988d1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vjhwc"
Jul 07 11:39:12.108513 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.108401    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b0afd6c-1e2a-4dee-b8fd-cde077988d1e-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vjhwc\" (UID: \"5b0afd6c-1e2a-4dee-b8fd-cde077988d1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vjhwc"
Jul 07 11:39:12.108513 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.108443    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b0afd6c-1e2a-4dee-b8fd-cde077988d1e-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vjhwc\" (UID: \"5b0afd6c-1e2a-4dee-b8fd-cde077988d1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vjhwc"
Jul 07 11:39:12.108513 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.108484    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b0afd6c-1e2a-4dee-b8fd-cde077988d1e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-vjhwc\" (UID: \"5b0afd6c-1e2a-4dee-b8fd-cde077988d1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vjhwc"
Jul 07 11:39:12.208895 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.208865    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b0afd6c-1e2a-4dee-b8fd-cde077988d1e-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vjhwc\" (UID: \"5b0afd6c-1e2a-4dee-b8fd-cde077988d1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vjhwc"
Jul 07 11:39:12.209088 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.208910    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b0afd6c-1e2a-4dee-b8fd-cde077988d1e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-vjhwc\" (UID: \"5b0afd6c-1e2a-4dee-b8fd-cde077988d1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vjhwc"
Jul 07 11:39:12.209088 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.208943    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b0afd6c-1e2a-4dee-b8fd-cde077988d1e-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-vjhwc\" (UID: \"5b0afd6c-1e2a-4dee-b8fd-cde077988d1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vjhwc"
Jul 07 11:39:12.209088 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.208981    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b0afd6c-1e2a-4dee-b8fd-cde077988d1e-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-vjhwc\" (UID: \"5b0afd6c-1e2a-4dee-b8fd-cde077988d1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vjhwc"
Jul 07 11:39:12.209088 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.209008    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b0afd6c-1e2a-4dee-b8fd-cde077988d1e-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-vjhwc\" (UID: \"5b0afd6c-1e2a-4dee-b8fd-cde077988d1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vjhwc"
Jul 07 11:39:12.209088 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.209059    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b0afd6c-1e2a-4dee-b8fd-cde077988d1e-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-vjhwc\" (UID: \"5b0afd6c-1e2a-4dee-b8fd-cde077988d1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vjhwc"
Jul 07 11:39:12.209376 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.209175    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b0afd6c-1e2a-4dee-b8fd-cde077988d1e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-vjhwc\" (UID: \"5b0afd6c-1e2a-4dee-b8fd-cde077988d1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vjhwc"
Jul 07 11:39:12.209376 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.209279    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdvs5\" (UniqueName: \"kubernetes.io/projected/5b0afd6c-1e2a-4dee-b8fd-cde077988d1e-kube-api-access-sdvs5\") pod \"maas-default-gateway-openshift-default-56654dcf7-vjhwc\" (UID: \"5b0afd6c-1e2a-4dee-b8fd-cde077988d1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vjhwc"
Jul 07 11:39:12.209376 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.209315    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b0afd6c-1e2a-4dee-b8fd-cde077988d1e-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vjhwc\" (UID: \"5b0afd6c-1e2a-4dee-b8fd-cde077988d1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vjhwc"
Jul 07 11:39:12.209376 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.209365    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b0afd6c-1e2a-4dee-b8fd-cde077988d1e-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vjhwc\" (UID: \"5b0afd6c-1e2a-4dee-b8fd-cde077988d1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vjhwc"
Jul 07 11:39:12.209555 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.209524    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b0afd6c-1e2a-4dee-b8fd-cde077988d1e-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-vjhwc\" (UID: \"5b0afd6c-1e2a-4dee-b8fd-cde077988d1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vjhwc"
Jul 07 11:39:12.209743 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.209719    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b0afd6c-1e2a-4dee-b8fd-cde077988d1e-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-vjhwc\" (UID: \"5b0afd6c-1e2a-4dee-b8fd-cde077988d1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vjhwc"
Jul 07 11:39:12.209902 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.209877    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b0afd6c-1e2a-4dee-b8fd-cde077988d1e-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vjhwc\" (UID: \"5b0afd6c-1e2a-4dee-b8fd-cde077988d1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vjhwc"
Jul 07 11:39:12.210143 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.210126    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b0afd6c-1e2a-4dee-b8fd-cde077988d1e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-vjhwc\" (UID: \"5b0afd6c-1e2a-4dee-b8fd-cde077988d1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vjhwc"
Jul 07 11:39:12.211288 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.211261    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b0afd6c-1e2a-4dee-b8fd-cde077988d1e-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-vjhwc\" (UID: \"5b0afd6c-1e2a-4dee-b8fd-cde077988d1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vjhwc"
Jul 07 11:39:12.211624 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.211604    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b0afd6c-1e2a-4dee-b8fd-cde077988d1e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-vjhwc\" (UID: \"5b0afd6c-1e2a-4dee-b8fd-cde077988d1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vjhwc"
Jul 07 11:39:12.217900 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.217873    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b0afd6c-1e2a-4dee-b8fd-cde077988d1e-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-vjhwc\" (UID: \"5b0afd6c-1e2a-4dee-b8fd-cde077988d1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vjhwc"
Jul 07 11:39:12.218045 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.218027    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdvs5\" (UniqueName: \"kubernetes.io/projected/5b0afd6c-1e2a-4dee-b8fd-cde077988d1e-kube-api-access-sdvs5\") pod \"maas-default-gateway-openshift-default-56654dcf7-vjhwc\" (UID: \"5b0afd6c-1e2a-4dee-b8fd-cde077988d1e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vjhwc"
Jul 07 11:39:12.342872 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.342794    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vjhwc"
Jul 07 11:39:12.486220 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.486181    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vjhwc"]
Jul 07 11:39:12.488234 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:39:12.488190    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b0afd6c_1e2a_4dee_b8fd_cde077988d1e.slice/crio-4ed734345b6dd0843df8bb2bdc593cae8d8a8ebe6bf9a78d471d8d3989aa772c WatchSource:0}: Error finding container 4ed734345b6dd0843df8bb2bdc593cae8d8a8ebe6bf9a78d471d8d3989aa772c: Status 404 returned error can't find the container with id 4ed734345b6dd0843df8bb2bdc593cae8d8a8ebe6bf9a78d471d8d3989aa772c
Jul 07 11:39:12.490661 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.490624    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 11:39:12.490761 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.490718    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 11:39:12.490761 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.490749    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 11:39:12.815523 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.815484    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vjhwc" event={"ID":"5b0afd6c-1e2a-4dee-b8fd-cde077988d1e","Type":"ContainerStarted","Data":"4c8c3996ece1fb125a4a3d33af1c2e5f99a9a6fe4051892ea6a04c52851eac89"}
Jul 07 11:39:12.815716 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.815540    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vjhwc" event={"ID":"5b0afd6c-1e2a-4dee-b8fd-cde077988d1e","Type":"ContainerStarted","Data":"4ed734345b6dd0843df8bb2bdc593cae8d8a8ebe6bf9a78d471d8d3989aa772c"}
Jul 07 11:39:12.837039 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:12.836989    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vjhwc" podStartSLOduration=0.836974456 podStartE2EDuration="836.974456ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:39:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:39:12.835971645 +0000 UTC m=+740.809612055" watchObservedRunningTime="2026-07-07 11:39:12.836974456 +0000 UTC m=+740.810614866"
Jul 07 11:39:13.343606 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:13.343572    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vjhwc"
Jul 07 11:39:13.348461 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:13.348439    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vjhwc"
Jul 07 11:39:13.821000 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:13.820971    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vjhwc"
Jul 07 11:39:13.822022 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:13.822003    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vjhwc"
Jul 07 11:39:27.330339 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:27.330297    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-d6ct4"]
Jul 07 11:39:27.333797 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:27.333775    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-d6ct4"
Jul 07 11:39:27.336676 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:27.336657    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-qxgqn\""
Jul 07 11:39:27.337876 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:27.337854    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 07 11:39:27.338017 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:27.337994    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 07 11:39:27.340609 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:27.340587    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-d6ct4"]
Jul 07 11:39:27.442670 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:27.442638    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sr4mh\" (UniqueName: \"kubernetes.io/projected/c14771c3-d35f-46e6-a734-4ad997d9f179-kube-api-access-sr4mh\") pod \"authorino-75d556bd77-d6ct4\" (UID: \"c14771c3-d35f-46e6-a734-4ad997d9f179\") " pod="kuadrant-system/authorino-75d556bd77-d6ct4"
Jul 07 11:39:27.543619 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:27.543587    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr4mh\" (UniqueName: \"kubernetes.io/projected/c14771c3-d35f-46e6-a734-4ad997d9f179-kube-api-access-sr4mh\") pod \"authorino-75d556bd77-d6ct4\" (UID: \"c14771c3-d35f-46e6-a734-4ad997d9f179\") " pod="kuadrant-system/authorino-75d556bd77-d6ct4"
Jul 07 11:39:27.552725 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:27.552697    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sr4mh\" (UniqueName: \"kubernetes.io/projected/c14771c3-d35f-46e6-a734-4ad997d9f179-kube-api-access-sr4mh\") pod \"authorino-75d556bd77-d6ct4\" (UID: \"c14771c3-d35f-46e6-a734-4ad997d9f179\") " pod="kuadrant-system/authorino-75d556bd77-d6ct4"
Jul 07 11:39:27.645127 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:27.645057    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-d6ct4"
Jul 07 11:39:27.777846 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:27.777817    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-d6ct4"]
Jul 07 11:39:27.779420 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:39:27.779393    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc14771c3_d35f_46e6_a734_4ad997d9f179.slice/crio-0598116761ed88c73face0c3e9ee775bf7bb32b4b42df1fd8cb303b5d37c37ba WatchSource:0}: Error finding container 0598116761ed88c73face0c3e9ee775bf7bb32b4b42df1fd8cb303b5d37c37ba: Status 404 returned error can't find the container with id 0598116761ed88c73face0c3e9ee775bf7bb32b4b42df1fd8cb303b5d37c37ba
Jul 07 11:39:27.889569 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:27.889534    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-d6ct4" event={"ID":"c14771c3-d35f-46e6-a734-4ad997d9f179","Type":"ContainerStarted","Data":"0598116761ed88c73face0c3e9ee775bf7bb32b4b42df1fd8cb303b5d37c37ba"}
Jul 07 11:39:30.907470 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:30.907381    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-d6ct4" event={"ID":"c14771c3-d35f-46e6-a734-4ad997d9f179","Type":"ContainerStarted","Data":"283b8be378607807fc8238767f5c3ae6b769eec9a3ddf781f720e7da7fdeac4a"}
Jul 07 11:39:30.922518 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:39:30.922466    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-d6ct4" podStartSLOduration=1.067625181 podStartE2EDuration="3.922448171s" totalImagesPullingTime="2.85482299s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:39:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:39:30.922335851 +0000 UTC m=+758.895976287" watchObservedRunningTime="2026-07-07 11:39:30.922448171 +0000 UTC m=+758.896088583"
Jul 07 11:40:00.009158 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:40:00.009121    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-d6ct4"]
Jul 07 11:40:00.009674 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:40:00.009336    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-d6ct4" podUID="c14771c3-d35f-46e6-a734-4ad997d9f179" containerName="authorino" containerID="cri-o://283b8be378607807fc8238767f5c3ae6b769eec9a3ddf781f720e7da7fdeac4a" gracePeriod=30
Jul 07 11:40:00.264866 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:40:00.264797    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-d6ct4"
Jul 07 11:40:00.332441 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:40:00.332399    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sr4mh\" (UniqueName: \"kubernetes.io/projected/c14771c3-d35f-46e6-a734-4ad997d9f179-kube-api-access-sr4mh\") pod \"c14771c3-d35f-46e6-a734-4ad997d9f179\" (UID: \"c14771c3-d35f-46e6-a734-4ad997d9f179\") "
Jul 07 11:40:00.334620 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:40:00.334589    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c14771c3-d35f-46e6-a734-4ad997d9f179-kube-api-access-sr4mh" (OuterVolumeSpecName: "kube-api-access-sr4mh") pod "c14771c3-d35f-46e6-a734-4ad997d9f179" (UID: "c14771c3-d35f-46e6-a734-4ad997d9f179"). InnerVolumeSpecName "kube-api-access-sr4mh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:40:00.433729 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:40:00.433690    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sr4mh\" (UniqueName: \"kubernetes.io/projected/c14771c3-d35f-46e6-a734-4ad997d9f179-kube-api-access-sr4mh\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:40:01.056881 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:40:01.056838    2570 generic.go:356] "Generic (PLEG): container finished" podID="c14771c3-d35f-46e6-a734-4ad997d9f179" containerID="283b8be378607807fc8238767f5c3ae6b769eec9a3ddf781f720e7da7fdeac4a" exitCode=0
Jul 07 11:40:01.057406 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:40:01.056891    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-d6ct4"
Jul 07 11:40:01.057406 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:40:01.056923    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-d6ct4" event={"ID":"c14771c3-d35f-46e6-a734-4ad997d9f179","Type":"ContainerDied","Data":"283b8be378607807fc8238767f5c3ae6b769eec9a3ddf781f720e7da7fdeac4a"}
Jul 07 11:40:01.057406 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:40:01.056965    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-d6ct4" event={"ID":"c14771c3-d35f-46e6-a734-4ad997d9f179","Type":"ContainerDied","Data":"0598116761ed88c73face0c3e9ee775bf7bb32b4b42df1fd8cb303b5d37c37ba"}
Jul 07 11:40:01.057406 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:40:01.056983    2570 scope.go:117] "RemoveContainer" containerID="283b8be378607807fc8238767f5c3ae6b769eec9a3ddf781f720e7da7fdeac4a"
Jul 07 11:40:01.068571 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:40:01.068546    2570 scope.go:117] "RemoveContainer" containerID="283b8be378607807fc8238767f5c3ae6b769eec9a3ddf781f720e7da7fdeac4a"
Jul 07 11:40:01.068922 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:40:01.068896    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"283b8be378607807fc8238767f5c3ae6b769eec9a3ddf781f720e7da7fdeac4a\": container with ID starting with 283b8be378607807fc8238767f5c3ae6b769eec9a3ddf781f720e7da7fdeac4a not found: ID does not exist" containerID="283b8be378607807fc8238767f5c3ae6b769eec9a3ddf781f720e7da7fdeac4a"
Jul 07 11:40:01.069023 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:40:01.068931    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"283b8be378607807fc8238767f5c3ae6b769eec9a3ddf781f720e7da7fdeac4a"} err="failed to get container status \"283b8be378607807fc8238767f5c3ae6b769eec9a3ddf781f720e7da7fdeac4a\": rpc error: code = NotFound desc = could not find container \"283b8be378607807fc8238767f5c3ae6b769eec9a3ddf781f720e7da7fdeac4a\": container with ID starting with 283b8be378607807fc8238767f5c3ae6b769eec9a3ddf781f720e7da7fdeac4a not found: ID does not exist"
Jul 07 11:40:01.080356 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:40:01.080320    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-d6ct4"]
Jul 07 11:40:01.082747 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:40:01.082724    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-d6ct4"]
Jul 07 11:40:01.938349 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:40:01.938316    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c14771c3-d35f-46e6-a734-4ad997d9f179" path="/var/lib/kubelet/pods/c14771c3-d35f-46e6-a734-4ad997d9f179/volumes"
Jul 07 11:40:44.863441 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:40:44.863400    2570 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-vjhwc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 07 11:40:44.866482 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:40:44.863477    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vjhwc" podUID="5b0afd6c-1e2a-4dee-b8fd-cde077988d1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 07 11:40:45.289703 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:40:45.289670    2570 generic.go:356] "Generic (PLEG): container finished" podID="5b0afd6c-1e2a-4dee-b8fd-cde077988d1e" containerID="4c8c3996ece1fb125a4a3d33af1c2e5f99a9a6fe4051892ea6a04c52851eac89" exitCode=0
Jul 07 11:40:45.289889 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:40:45.289746    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vjhwc" event={"ID":"5b0afd6c-1e2a-4dee-b8fd-cde077988d1e","Type":"ContainerDied","Data":"4c8c3996ece1fb125a4a3d33af1c2e5f99a9a6fe4051892ea6a04c52851eac89"}
Jul 07 11:40:45.290174 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:40:45.290160    2570 scope.go:117] "RemoveContainer" containerID="4c8c3996ece1fb125a4a3d33af1c2e5f99a9a6fe4051892ea6a04c52851eac89"
Jul 07 11:40:45.291139 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:40:45.291108    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 11:40:45.291281 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:40:45.291178    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 11:40:45.291281 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:40:45.291230    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 11:40:46.298073 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:40:46.298040    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vjhwc" event={"ID":"5b0afd6c-1e2a-4dee-b8fd-cde077988d1e","Type":"ContainerStarted","Data":"cdf46db1f2d5043b4c4580f33eeeff7a74ab00e8aa61e64a7181e298a3bfe3a2"}
Jul 07 11:40:46.343783 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:40:46.343747    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vjhwc"
Jul 07 11:40:46.344074 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:40:46.343804    2570 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-vjhwc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body=
Jul 07 11:40:46.344269 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:40:46.344243    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vjhwc" podUID="5b0afd6c-1e2a-4dee-b8fd-cde077988d1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused"
Jul 07 11:40:47.344146 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:40:47.344105    2570 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-vjhwc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body=
Jul 07 11:40:47.344814 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:40:47.344778    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vjhwc" podUID="5b0afd6c-1e2a-4dee-b8fd-cde077988d1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused"
Jul 07 11:40:48.344203 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:40:48.344151    2570 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-vjhwc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body=
Jul 07 11:40:48.344741 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:40:48.344250    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vjhwc" podUID="5b0afd6c-1e2a-4dee-b8fd-cde077988d1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused"
Jul 07 11:40:49.343638 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:40:49.343596    2570 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-vjhwc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body=
Jul 07 11:40:49.343954 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:40:49.343923    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vjhwc" podUID="5b0afd6c-1e2a-4dee-b8fd-cde077988d1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused"
Jul 07 11:40:50.347411 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:40:50.347379    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vjhwc"
Jul 07 11:40:50.347816 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:40:50.347726    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vjhwc"
Jul 07 11:40:50.348369 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:40:50.348349    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vjhwc"
Jul 07 11:41:20.516020 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:20.515984    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-5kn7q"]
Jul 07 11:41:20.516535 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:20.516444    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c14771c3-d35f-46e6-a734-4ad997d9f179" containerName="authorino"
Jul 07 11:41:20.516535 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:20.516460    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="c14771c3-d35f-46e6-a734-4ad997d9f179" containerName="authorino"
Jul 07 11:41:20.516535 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:20.516526    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="c14771c3-d35f-46e6-a734-4ad997d9f179" containerName="authorino"
Jul 07 11:41:20.519536 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:20.519519    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kn7q"
Jul 07 11:41:20.522587 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:20.522562    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 07 11:41:20.523739 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:20.523713    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 07 11:41:20.523881 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:20.523752    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-9wjpf\""
Jul 07 11:41:20.523881 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:20.523752    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 07 11:41:20.535882 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:20.535843    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-5kn7q"]
Jul 07 11:41:20.646667 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:20.646621    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3838b1cc-3390-449b-b705-97fbb6c5caf2-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kn7q\" (UID: \"3838b1cc-3390-449b-b705-97fbb6c5caf2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kn7q"
Jul 07 11:41:20.646667 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:20.646669    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zvdc\" (UniqueName: \"kubernetes.io/projected/3838b1cc-3390-449b-b705-97fbb6c5caf2-kube-api-access-5zvdc\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kn7q\" (UID: \"3838b1cc-3390-449b-b705-97fbb6c5caf2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kn7q"
Jul 07 11:41:20.646912 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:20.646786    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3838b1cc-3390-449b-b705-97fbb6c5caf2-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kn7q\" (UID: \"3838b1cc-3390-449b-b705-97fbb6c5caf2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kn7q"
Jul 07 11:41:20.646912 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:20.646828    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3838b1cc-3390-449b-b705-97fbb6c5caf2-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kn7q\" (UID: \"3838b1cc-3390-449b-b705-97fbb6c5caf2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kn7q"
Jul 07 11:41:20.646993 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:20.646907    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3838b1cc-3390-449b-b705-97fbb6c5caf2-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kn7q\" (UID: \"3838b1cc-3390-449b-b705-97fbb6c5caf2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kn7q"
Jul 07 11:41:20.646993 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:20.646948    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3838b1cc-3390-449b-b705-97fbb6c5caf2-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kn7q\" (UID: \"3838b1cc-3390-449b-b705-97fbb6c5caf2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kn7q"
Jul 07 11:41:20.748000 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:20.747953    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3838b1cc-3390-449b-b705-97fbb6c5caf2-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kn7q\" (UID: \"3838b1cc-3390-449b-b705-97fbb6c5caf2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kn7q"
Jul 07 11:41:20.748000 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:20.748000    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3838b1cc-3390-449b-b705-97fbb6c5caf2-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kn7q\" (UID: \"3838b1cc-3390-449b-b705-97fbb6c5caf2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kn7q"
Jul 07 11:41:20.748276 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:20.748021    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zvdc\" (UniqueName: \"kubernetes.io/projected/3838b1cc-3390-449b-b705-97fbb6c5caf2-kube-api-access-5zvdc\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kn7q\" (UID: \"3838b1cc-3390-449b-b705-97fbb6c5caf2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kn7q"
Jul 07 11:41:20.748276 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:20.748088    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3838b1cc-3390-449b-b705-97fbb6c5caf2-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kn7q\" (UID: \"3838b1cc-3390-449b-b705-97fbb6c5caf2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kn7q"
Jul 07 11:41:20.748276 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:20.748117    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3838b1cc-3390-449b-b705-97fbb6c5caf2-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kn7q\" (UID: \"3838b1cc-3390-449b-b705-97fbb6c5caf2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kn7q"
Jul 07 11:41:20.748276 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:20.748163    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3838b1cc-3390-449b-b705-97fbb6c5caf2-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kn7q\" (UID: \"3838b1cc-3390-449b-b705-97fbb6c5caf2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kn7q"
Jul 07 11:41:20.748517 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:20.748436    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3838b1cc-3390-449b-b705-97fbb6c5caf2-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kn7q\" (UID: \"3838b1cc-3390-449b-b705-97fbb6c5caf2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kn7q"
Jul 07 11:41:20.748517 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:20.748486    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3838b1cc-3390-449b-b705-97fbb6c5caf2-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kn7q\" (UID: \"3838b1cc-3390-449b-b705-97fbb6c5caf2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kn7q"
Jul 07 11:41:20.748630 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:20.748520    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3838b1cc-3390-449b-b705-97fbb6c5caf2-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kn7q\" (UID: \"3838b1cc-3390-449b-b705-97fbb6c5caf2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kn7q"
Jul 07 11:41:20.750466 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:20.750428    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3838b1cc-3390-449b-b705-97fbb6c5caf2-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kn7q\" (UID: \"3838b1cc-3390-449b-b705-97fbb6c5caf2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kn7q"
Jul 07 11:41:20.750699 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:20.750682    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3838b1cc-3390-449b-b705-97fbb6c5caf2-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kn7q\" (UID: \"3838b1cc-3390-449b-b705-97fbb6c5caf2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kn7q"
Jul 07 11:41:20.756912 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:20.756885    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zvdc\" (UniqueName: \"kubernetes.io/projected/3838b1cc-3390-449b-b705-97fbb6c5caf2-kube-api-access-5zvdc\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5kn7q\" (UID: \"3838b1cc-3390-449b-b705-97fbb6c5caf2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kn7q"
Jul 07 11:41:20.831477 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:20.831375    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kn7q"
Jul 07 11:41:20.971938 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:20.971908    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-5kn7q"]
Jul 07 11:41:20.973693 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:41:20.973661    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3838b1cc_3390_449b_b705_97fbb6c5caf2.slice/crio-802429cf113124aba467a9c2b95f542df27700bd1c5dcdbb11c6d629d44835ae WatchSource:0}: Error finding container 802429cf113124aba467a9c2b95f542df27700bd1c5dcdbb11c6d629d44835ae: Status 404 returned error can't find the container with id 802429cf113124aba467a9c2b95f542df27700bd1c5dcdbb11c6d629d44835ae
Jul 07 11:41:21.483134 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:21.483092    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kn7q" event={"ID":"3838b1cc-3390-449b-b705-97fbb6c5caf2","Type":"ContainerStarted","Data":"802429cf113124aba467a9c2b95f542df27700bd1c5dcdbb11c6d629d44835ae"}
Jul 07 11:41:22.489206 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:22.489166    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kn7q" event={"ID":"3838b1cc-3390-449b-b705-97fbb6c5caf2","Type":"ContainerStarted","Data":"8fd6e07711d8ed0a6d4530567cae62d950f4c39b8c5c00a228d2bd11e05d6b67"}
Jul 07 11:41:22.508914 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:22.508857    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kn7q" podStartSLOduration=1.48531862 podStartE2EDuration="2.508840434s" totalImagesPullingTime="1.023521814s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:41:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:41:22.506826969 +0000 UTC m=+870.480467415" watchObservedRunningTime="2026-07-07 11:41:22.508840434 +0000 UTC m=+870.482480845"
Jul 07 11:41:29.226776 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:29.226724    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrbbwp"]
Jul 07 11:41:29.230407 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:29.230387    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrbbwp"
Jul 07 11:41:29.233992 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:29.233958    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 07 11:41:29.241184 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:29.241155    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrbbwp"]
Jul 07 11:41:29.326403 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:29.326362    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14348839-2afe-4987-84c9-46ed1f0bb1a7-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrbbwp\" (UID: \"14348839-2afe-4987-84c9-46ed1f0bb1a7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrbbwp"
Jul 07 11:41:29.326597 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:29.326427    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfnnc\" (UniqueName: \"kubernetes.io/projected/14348839-2afe-4987-84c9-46ed1f0bb1a7-kube-api-access-bfnnc\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrbbwp\" (UID: \"14348839-2afe-4987-84c9-46ed1f0bb1a7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrbbwp"
Jul 07 11:41:29.326597 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:29.326446    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/14348839-2afe-4987-84c9-46ed1f0bb1a7-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrbbwp\" (UID: \"14348839-2afe-4987-84c9-46ed1f0bb1a7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrbbwp"
Jul 07 11:41:29.326597 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:29.326503    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/14348839-2afe-4987-84c9-46ed1f0bb1a7-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrbbwp\" (UID: \"14348839-2afe-4987-84c9-46ed1f0bb1a7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrbbwp"
Jul 07 11:41:29.326597 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:29.326542    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/14348839-2afe-4987-84c9-46ed1f0bb1a7-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrbbwp\" (UID: \"14348839-2afe-4987-84c9-46ed1f0bb1a7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrbbwp"
Jul 07 11:41:29.326597 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:29.326577    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14348839-2afe-4987-84c9-46ed1f0bb1a7-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrbbwp\" (UID: \"14348839-2afe-4987-84c9-46ed1f0bb1a7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrbbwp"
Jul 07 11:41:29.428045 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:29.428004    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/14348839-2afe-4987-84c9-46ed1f0bb1a7-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrbbwp\" (UID: \"14348839-2afe-4987-84c9-46ed1f0bb1a7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrbbwp"
Jul 07 11:41:29.428272 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:29.428051    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/14348839-2afe-4987-84c9-46ed1f0bb1a7-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrbbwp\" (UID: \"14348839-2afe-4987-84c9-46ed1f0bb1a7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrbbwp"
Jul 07 11:41:29.428272 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:29.428094    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14348839-2afe-4987-84c9-46ed1f0bb1a7-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrbbwp\" (UID: \"14348839-2afe-4987-84c9-46ed1f0bb1a7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrbbwp"
Jul 07 11:41:29.428414 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:29.428265    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14348839-2afe-4987-84c9-46ed1f0bb1a7-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrbbwp\" (UID: \"14348839-2afe-4987-84c9-46ed1f0bb1a7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrbbwp"
Jul 07 11:41:29.428414 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:29.428381    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfnnc\" (UniqueName: \"kubernetes.io/projected/14348839-2afe-4987-84c9-46ed1f0bb1a7-kube-api-access-bfnnc\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrbbwp\" (UID: \"14348839-2afe-4987-84c9-46ed1f0bb1a7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrbbwp"
Jul 07 11:41:29.428531 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:29.428414    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/14348839-2afe-4987-84c9-46ed1f0bb1a7-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrbbwp\" (UID: \"14348839-2afe-4987-84c9-46ed1f0bb1a7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrbbwp"
Jul 07 11:41:29.428588 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:29.428539    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/14348839-2afe-4987-84c9-46ed1f0bb1a7-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrbbwp\" (UID: \"14348839-2afe-4987-84c9-46ed1f0bb1a7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrbbwp"
Jul 07 11:41:29.428625 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:29.428573    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14348839-2afe-4987-84c9-46ed1f0bb1a7-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrbbwp\" (UID: \"14348839-2afe-4987-84c9-46ed1f0bb1a7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrbbwp"
Jul 07 11:41:29.428749 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:29.428728    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/14348839-2afe-4987-84c9-46ed1f0bb1a7-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrbbwp\" (UID: \"14348839-2afe-4987-84c9-46ed1f0bb1a7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrbbwp"
Jul 07 11:41:29.430551 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:29.430522    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/14348839-2afe-4987-84c9-46ed1f0bb1a7-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrbbwp\" (UID: \"14348839-2afe-4987-84c9-46ed1f0bb1a7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrbbwp"
Jul 07 11:41:29.430743 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:29.430723    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14348839-2afe-4987-84c9-46ed1f0bb1a7-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrbbwp\" (UID: \"14348839-2afe-4987-84c9-46ed1f0bb1a7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrbbwp"
Jul 07 11:41:29.438126 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:29.438100    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfnnc\" (UniqueName: \"kubernetes.io/projected/14348839-2afe-4987-84c9-46ed1f0bb1a7-kube-api-access-bfnnc\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrbbwp\" (UID: \"14348839-2afe-4987-84c9-46ed1f0bb1a7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrbbwp"
Jul 07 11:41:29.541385 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:29.541291    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrbbwp"
Jul 07 11:41:29.680710 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:29.680678    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrbbwp"]
Jul 07 11:41:29.682124 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:41:29.682087    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14348839_2afe_4987_84c9_46ed1f0bb1a7.slice/crio-e780019088147bd247a889ebc9a05dded385b64f24db14415f11f6abd0748be1 WatchSource:0}: Error finding container e780019088147bd247a889ebc9a05dded385b64f24db14415f11f6abd0748be1: Status 404 returned error can't find the container with id e780019088147bd247a889ebc9a05dded385b64f24db14415f11f6abd0748be1
Jul 07 11:41:30.535816 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:30.535778    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrbbwp" event={"ID":"14348839-2afe-4987-84c9-46ed1f0bb1a7","Type":"ContainerStarted","Data":"f8e38b91dbfc07972158273349c4b75edc21acc066803c9d3bb22d2bddba3768"}
Jul 07 11:41:30.536325 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:30.535836    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrbbwp" event={"ID":"14348839-2afe-4987-84c9-46ed1f0bb1a7","Type":"ContainerStarted","Data":"e780019088147bd247a889ebc9a05dded385b64f24db14415f11f6abd0748be1"}
Jul 07 11:41:30.555610 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:30.555555    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrbbwp" podStartSLOduration=1.356524059 podStartE2EDuration="1.555536857s" totalImagesPullingTime="199.012798ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:41:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:41:30.555031444 +0000 UTC m=+878.528671856" watchObservedRunningTime="2026-07-07 11:41:30.555536857 +0000 UTC m=+878.529177266"
Jul 07 11:41:30.832191 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:30.832095    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kn7q"
Jul 07 11:41:30.832397 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:30.832183    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kn7q"
Jul 07 11:41:30.844818 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:30.844783    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kn7q"
Jul 07 11:41:31.552594 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:31.552551    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5kn7q"
Jul 07 11:41:39.542116 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:39.542075    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrbbwp"
Jul 07 11:41:39.542695 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:39.542168    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrbbwp"
Jul 07 11:41:39.555020 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:39.554987    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrbbwp"
Jul 07 11:41:39.592831 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:39.592799    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrbbwp"
Jul 07 11:41:43.619028 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:43.618999    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-45cgf"]
Jul 07 11:41:43.622352 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:43.622332    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-45cgf"
Jul 07 11:41:43.625138 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:43.625114    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 07 11:41:43.634421 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:43.634398    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-45cgf"]
Jul 07 11:41:43.763077 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:43.763039    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a19573b6-1ff3-4b0a-a47f-915c7aa2b490-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-45cgf\" (UID: \"a19573b6-1ff3-4b0a-a47f-915c7aa2b490\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-45cgf"
Jul 07 11:41:43.763319 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:43.763111    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a19573b6-1ff3-4b0a-a47f-915c7aa2b490-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-45cgf\" (UID: \"a19573b6-1ff3-4b0a-a47f-915c7aa2b490\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-45cgf"
Jul 07 11:41:43.763319 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:43.763188    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a19573b6-1ff3-4b0a-a47f-915c7aa2b490-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-45cgf\" (UID: \"a19573b6-1ff3-4b0a-a47f-915c7aa2b490\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-45cgf"
Jul 07 11:41:43.763319 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:43.763251    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a19573b6-1ff3-4b0a-a47f-915c7aa2b490-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-45cgf\" (UID: \"a19573b6-1ff3-4b0a-a47f-915c7aa2b490\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-45cgf"
Jul 07 11:41:43.763319 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:43.763302    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a19573b6-1ff3-4b0a-a47f-915c7aa2b490-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-45cgf\" (UID: \"a19573b6-1ff3-4b0a-a47f-915c7aa2b490\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-45cgf"
Jul 07 11:41:43.763466 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:43.763322    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqf6b\" (UniqueName: \"kubernetes.io/projected/a19573b6-1ff3-4b0a-a47f-915c7aa2b490-kube-api-access-sqf6b\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-45cgf\" (UID: \"a19573b6-1ff3-4b0a-a47f-915c7aa2b490\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-45cgf"
Jul 07 11:41:43.863790 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:43.863752    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a19573b6-1ff3-4b0a-a47f-915c7aa2b490-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-45cgf\" (UID: \"a19573b6-1ff3-4b0a-a47f-915c7aa2b490\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-45cgf"
Jul 07 11:41:43.863973 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:43.863806    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a19573b6-1ff3-4b0a-a47f-915c7aa2b490-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-45cgf\" (UID: \"a19573b6-1ff3-4b0a-a47f-915c7aa2b490\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-45cgf"
Jul 07 11:41:43.863973 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:43.863845    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a19573b6-1ff3-4b0a-a47f-915c7aa2b490-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-45cgf\" (UID: \"a19573b6-1ff3-4b0a-a47f-915c7aa2b490\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-45cgf"
Jul 07 11:41:43.863973 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:43.863884    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a19573b6-1ff3-4b0a-a47f-915c7aa2b490-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-45cgf\" (UID: \"a19573b6-1ff3-4b0a-a47f-915c7aa2b490\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-45cgf"
Jul 07 11:41:43.863973 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:43.863914    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqf6b\" (UniqueName: \"kubernetes.io/projected/a19573b6-1ff3-4b0a-a47f-915c7aa2b490-kube-api-access-sqf6b\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-45cgf\" (UID: \"a19573b6-1ff3-4b0a-a47f-915c7aa2b490\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-45cgf"
Jul 07 11:41:43.863973 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:43.863955    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a19573b6-1ff3-4b0a-a47f-915c7aa2b490-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-45cgf\" (UID: \"a19573b6-1ff3-4b0a-a47f-915c7aa2b490\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-45cgf"
Jul 07 11:41:43.864283 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:43.864251    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a19573b6-1ff3-4b0a-a47f-915c7aa2b490-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-45cgf\" (UID: \"a19573b6-1ff3-4b0a-a47f-915c7aa2b490\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-45cgf"
Jul 07 11:41:43.864425 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:43.864292    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a19573b6-1ff3-4b0a-a47f-915c7aa2b490-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-45cgf\" (UID: \"a19573b6-1ff3-4b0a-a47f-915c7aa2b490\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-45cgf"
Jul 07 11:41:43.864425 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:43.864333    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a19573b6-1ff3-4b0a-a47f-915c7aa2b490-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-45cgf\" (UID: \"a19573b6-1ff3-4b0a-a47f-915c7aa2b490\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-45cgf"
Jul 07 11:41:43.866199 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:43.866178    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a19573b6-1ff3-4b0a-a47f-915c7aa2b490-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-45cgf\" (UID: \"a19573b6-1ff3-4b0a-a47f-915c7aa2b490\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-45cgf"
Jul 07 11:41:43.866614 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:43.866593    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a19573b6-1ff3-4b0a-a47f-915c7aa2b490-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-45cgf\" (UID: \"a19573b6-1ff3-4b0a-a47f-915c7aa2b490\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-45cgf"
Jul 07 11:41:43.872090 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:43.872027    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqf6b\" (UniqueName: \"kubernetes.io/projected/a19573b6-1ff3-4b0a-a47f-915c7aa2b490-kube-api-access-sqf6b\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-45cgf\" (UID: \"a19573b6-1ff3-4b0a-a47f-915c7aa2b490\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-45cgf"
Jul 07 11:41:43.959317 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:43.959280    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-45cgf"
Jul 07 11:41:44.120737 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:44.120712    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-45cgf"]
Jul 07 11:41:44.121223 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:41:44.121182    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda19573b6_1ff3_4b0a_a47f_915c7aa2b490.slice/crio-bd49a15e6e1e60eca02752989666b8fc6084093d3c228cdfc22f4f78d9bc9ded WatchSource:0}: Error finding container bd49a15e6e1e60eca02752989666b8fc6084093d3c228cdfc22f4f78d9bc9ded: Status 404 returned error can't find the container with id bd49a15e6e1e60eca02752989666b8fc6084093d3c228cdfc22f4f78d9bc9ded
Jul 07 11:41:44.607107 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:44.607068    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-45cgf" event={"ID":"a19573b6-1ff3-4b0a-a47f-915c7aa2b490","Type":"ContainerStarted","Data":"266f5f59a6ecb7f7dd1cffbec11fdc86bde18006f95c967d1d468f6e750d66ca"}
Jul 07 11:41:44.607107 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:44.607117    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-45cgf" event={"ID":"a19573b6-1ff3-4b0a-a47f-915c7aa2b490","Type":"ContainerStarted","Data":"bd49a15e6e1e60eca02752989666b8fc6084093d3c228cdfc22f4f78d9bc9ded"}
Jul 07 11:41:44.627576 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:44.627466    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-45cgf" podStartSLOduration=1.424128032 podStartE2EDuration="1.627449064s" totalImagesPullingTime="203.321032ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:41:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:41:44.625837177 +0000 UTC m=+892.599477587" watchObservedRunningTime="2026-07-07 11:41:44.627449064 +0000 UTC m=+892.601089474"
Jul 07 11:41:53.893955 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:53.893926    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ktbpw_faf6f343-6515-4b2c-a34e-286cd2d9ec7a/console-operator/1.log"
Jul 07 11:41:53.894774 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:53.894758    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ktbpw_faf6f343-6515-4b2c-a34e-286cd2d9ec7a/console-operator/1.log"
Jul 07 11:41:53.959652 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:53.959616    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-45cgf"
Jul 07 11:41:53.959825 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:53.959668    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-45cgf"
Jul 07 11:41:53.972706 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:53.972679    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-45cgf"
Jul 07 11:41:54.668671 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:41:54.668640    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-45cgf"
Jul 07 11:46:53.927432 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:46:53.927352    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ktbpw_faf6f343-6515-4b2c-a34e-286cd2d9ec7a/console-operator/1.log"
Jul 07 11:46:53.930454 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:46:53.930428    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ktbpw_faf6f343-6515-4b2c-a34e-286cd2d9ec7a/console-operator/1.log"
Jul 07 11:51:53.968053 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:51:53.968025    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ktbpw_faf6f343-6515-4b2c-a34e-286cd2d9ec7a/console-operator/1.log"
Jul 07 11:51:53.975263 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:51:53.975238    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ktbpw_faf6f343-6515-4b2c-a34e-286cd2d9ec7a/console-operator/1.log"
Jul 07 11:56:06.212120 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:06.212088    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx"]
Jul 07 11:56:06.215666 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:06.215644    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx"
Jul 07 11:56:06.220175 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:06.220153    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-044dd768-openshift-default-dockercfg-4f7vm\""
Jul 07 11:56:06.256881 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:06.256847    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx"]
Jul 07 11:56:06.362270 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:06.362199    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-istio-data\") pod \"e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx\" (UID: \"21d92c25-b83a-4f6c-a18d-d2e54eb96f09\") " pod="openshift-ingress/e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx"
Jul 07 11:56:06.362449 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:06.362287    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-istiod-ca-cert\") pod \"e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx\" (UID: \"21d92c25-b83a-4f6c-a18d-d2e54eb96f09\") " pod="openshift-ingress/e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx"
Jul 07 11:56:06.362449 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:06.362311    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-istio-token\") pod \"e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx\" (UID: \"21d92c25-b83a-4f6c-a18d-d2e54eb96f09\") " pod="openshift-ingress/e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx"
Jul 07 11:56:06.362449 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:06.362326    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p65p6\" (UniqueName: \"kubernetes.io/projected/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-kube-api-access-p65p6\") pod \"e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx\" (UID: \"21d92c25-b83a-4f6c-a18d-d2e54eb96f09\") " pod="openshift-ingress/e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx"
Jul 07 11:56:06.362449 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:06.362394    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-credential-socket\") pod \"e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx\" (UID: \"21d92c25-b83a-4f6c-a18d-d2e54eb96f09\") " pod="openshift-ingress/e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx"
Jul 07 11:56:06.362449 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:06.362413    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-istio-envoy\") pod \"e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx\" (UID: \"21d92c25-b83a-4f6c-a18d-d2e54eb96f09\") " pod="openshift-ingress/e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx"
Jul 07 11:56:06.362616 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:06.362519    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-workload-socket\") pod \"e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx\" (UID: \"21d92c25-b83a-4f6c-a18d-d2e54eb96f09\") " pod="openshift-ingress/e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx"
Jul 07 11:56:06.362616 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:06.362553    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-workload-certs\") pod \"e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx\" (UID: \"21d92c25-b83a-4f6c-a18d-d2e54eb96f09\") " pod="openshift-ingress/e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx"
Jul 07 11:56:06.362616 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:06.362574    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-istio-podinfo\") pod \"e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx\" (UID: \"21d92c25-b83a-4f6c-a18d-d2e54eb96f09\") " pod="openshift-ingress/e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx"
Jul 07 11:56:06.463982 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:06.463885    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-workload-socket\") pod \"e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx\" (UID: \"21d92c25-b83a-4f6c-a18d-d2e54eb96f09\") " pod="openshift-ingress/e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx"
Jul 07 11:56:06.463982 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:06.463932    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-workload-certs\") pod \"e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx\" (UID: \"21d92c25-b83a-4f6c-a18d-d2e54eb96f09\") " pod="openshift-ingress/e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx"
Jul 07 11:56:06.463982 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:06.463958    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-istio-podinfo\") pod \"e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx\" (UID: \"21d92c25-b83a-4f6c-a18d-d2e54eb96f09\") " pod="openshift-ingress/e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx"
Jul 07 11:56:06.464322 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:06.463999    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-istio-data\") pod \"e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx\" (UID: \"21d92c25-b83a-4f6c-a18d-d2e54eb96f09\") " pod="openshift-ingress/e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx"
Jul 07 11:56:06.464322 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:06.464133    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-istiod-ca-cert\") pod \"e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx\" (UID: \"21d92c25-b83a-4f6c-a18d-d2e54eb96f09\") " pod="openshift-ingress/e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx"
Jul 07 11:56:06.464322 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:06.464170    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-istio-token\") pod \"e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx\" (UID: \"21d92c25-b83a-4f6c-a18d-d2e54eb96f09\") " pod="openshift-ingress/e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx"
Jul 07 11:56:06.464322 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:06.464187    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p65p6\" (UniqueName: \"kubernetes.io/projected/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-kube-api-access-p65p6\") pod \"e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx\" (UID: \"21d92c25-b83a-4f6c-a18d-d2e54eb96f09\") " pod="openshift-ingress/e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx"
Jul 07 11:56:06.464322 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:06.464286    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-credential-socket\") pod \"e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx\" (UID: \"21d92c25-b83a-4f6c-a18d-d2e54eb96f09\") " pod="openshift-ingress/e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx"
Jul 07 11:56:06.464322 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:06.464314    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-istio-envoy\") pod \"e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx\" (UID: \"21d92c25-b83a-4f6c-a18d-d2e54eb96f09\") " pod="openshift-ingress/e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx"
Jul 07 11:56:06.464642 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:06.464345    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-workload-socket\") pod \"e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx\" (UID: \"21d92c25-b83a-4f6c-a18d-d2e54eb96f09\") " pod="openshift-ingress/e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx"
Jul 07 11:56:06.464642 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:06.464385    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-istio-data\") pod \"e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx\" (UID: \"21d92c25-b83a-4f6c-a18d-d2e54eb96f09\") " pod="openshift-ingress/e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx"
Jul 07 11:56:06.464642 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:06.464443    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-workload-certs\") pod \"e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx\" (UID: \"21d92c25-b83a-4f6c-a18d-d2e54eb96f09\") " pod="openshift-ingress/e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx"
Jul 07 11:56:06.464758 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:06.464668    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-credential-socket\") pod \"e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx\" (UID: \"21d92c25-b83a-4f6c-a18d-d2e54eb96f09\") " pod="openshift-ingress/e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx"
Jul 07 11:56:06.464942 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:06.464919    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-istiod-ca-cert\") pod \"e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx\" (UID: \"21d92c25-b83a-4f6c-a18d-d2e54eb96f09\") " pod="openshift-ingress/e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx"
Jul 07 11:56:06.466458 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:06.466438    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-istio-envoy\") pod \"e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx\" (UID: \"21d92c25-b83a-4f6c-a18d-d2e54eb96f09\") " pod="openshift-ingress/e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx"
Jul 07 11:56:06.466807 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:06.466785    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-istio-podinfo\") pod \"e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx\" (UID: \"21d92c25-b83a-4f6c-a18d-d2e54eb96f09\") " pod="openshift-ingress/e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx"
Jul 07 11:56:06.473135 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:06.473109    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p65p6\" (UniqueName: \"kubernetes.io/projected/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-kube-api-access-p65p6\") pod \"e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx\" (UID: \"21d92c25-b83a-4f6c-a18d-d2e54eb96f09\") " pod="openshift-ingress/e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx"
Jul 07 11:56:06.473824 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:06.473808    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-istio-token\") pod \"e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx\" (UID: \"21d92c25-b83a-4f6c-a18d-d2e54eb96f09\") " pod="openshift-ingress/e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx"
Jul 07 11:56:06.525624 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:06.525587    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx"
Jul 07 11:56:06.667781 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:06.667750    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx"]
Jul 07 11:56:06.668536 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:56:06.668507    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21d92c25_b83a_4f6c_a18d_d2e54eb96f09.slice/crio-4e47d20b96473b089a2932014e24d1bf144dea1c1b90a38988a6984d93931272 WatchSource:0}: Error finding container 4e47d20b96473b089a2932014e24d1bf144dea1c1b90a38988a6984d93931272: Status 404 returned error can't find the container with id 4e47d20b96473b089a2932014e24d1bf144dea1c1b90a38988a6984d93931272
Jul 07 11:56:06.670832 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:06.670801    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 11:56:06.670922 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:06.670873    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 11:56:06.670922 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:06.670901    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 11:56:07.039902 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:07.039799    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx" event={"ID":"21d92c25-b83a-4f6c-a18d-d2e54eb96f09","Type":"ContainerStarted","Data":"b4e4c1abe1fcbc8c7d746ab263841a8e2bf781d32bead53e7d00644deff8763b"}
Jul 07 11:56:07.039902 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:07.039859    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx" event={"ID":"21d92c25-b83a-4f6c-a18d-d2e54eb96f09","Type":"ContainerStarted","Data":"4e47d20b96473b089a2932014e24d1bf144dea1c1b90a38988a6984d93931272"}
Jul 07 11:56:07.062418 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:07.062368    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx" podStartSLOduration=1.062351111 podStartE2EDuration="1.062351111s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:56:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:56:07.060747358 +0000 UTC m=+1755.034387804" watchObservedRunningTime="2026-07-07 11:56:07.062351111 +0000 UTC m=+1755.035991528"
Jul 07 11:56:07.526608 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:07.526568    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx"
Jul 07 11:56:07.531382 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:07.531356    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx"
Jul 07 11:56:08.045304 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:08.045272    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx"
Jul 07 11:56:08.046352 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:08.046330    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx"
Jul 07 11:56:12.631506 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:12.631434    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx"]
Jul 07 11:56:12.632051 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:12.631750    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx" podUID="21d92c25-b83a-4f6c-a18d-d2e54eb96f09" containerName="istio-proxy" containerID="cri-o://b4e4c1abe1fcbc8c7d746ab263841a8e2bf781d32bead53e7d00644deff8763b" gracePeriod=30
Jul 07 11:56:18.686524 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:18.686497    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx"
Jul 07 11:56:18.792464 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:18.792371    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-credential-socket\") pod \"21d92c25-b83a-4f6c-a18d-d2e54eb96f09\" (UID: \"21d92c25-b83a-4f6c-a18d-d2e54eb96f09\") "
Jul 07 11:56:18.792464 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:18.792417    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-workload-socket\") pod \"21d92c25-b83a-4f6c-a18d-d2e54eb96f09\" (UID: \"21d92c25-b83a-4f6c-a18d-d2e54eb96f09\") "
Jul 07 11:56:18.792464 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:18.792447    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p65p6\" (UniqueName: \"kubernetes.io/projected/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-kube-api-access-p65p6\") pod \"21d92c25-b83a-4f6c-a18d-d2e54eb96f09\" (UID: \"21d92c25-b83a-4f6c-a18d-d2e54eb96f09\") "
Jul 07 11:56:18.792759 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:18.792481    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-istio-envoy\") pod \"21d92c25-b83a-4f6c-a18d-d2e54eb96f09\" (UID: \"21d92c25-b83a-4f6c-a18d-d2e54eb96f09\") "
Jul 07 11:56:18.792759 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:18.792515    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-istiod-ca-cert\") pod \"21d92c25-b83a-4f6c-a18d-d2e54eb96f09\" (UID: \"21d92c25-b83a-4f6c-a18d-d2e54eb96f09\") "
Jul 07 11:56:18.792759 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:18.792696    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-istio-token\") pod \"21d92c25-b83a-4f6c-a18d-d2e54eb96f09\" (UID: \"21d92c25-b83a-4f6c-a18d-d2e54eb96f09\") "
Jul 07 11:56:18.792759 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:18.792715    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "21d92c25-b83a-4f6c-a18d-d2e54eb96f09" (UID: "21d92c25-b83a-4f6c-a18d-d2e54eb96f09"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:56:18.792759 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:18.792751    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-workload-certs\") pod \"21d92c25-b83a-4f6c-a18d-d2e54eb96f09\" (UID: \"21d92c25-b83a-4f6c-a18d-d2e54eb96f09\") "
Jul 07 11:56:18.793018 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:18.792754    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "21d92c25-b83a-4f6c-a18d-d2e54eb96f09" (UID: "21d92c25-b83a-4f6c-a18d-d2e54eb96f09"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:56:18.793018 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:18.792786    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-istio-podinfo\") pod \"21d92c25-b83a-4f6c-a18d-d2e54eb96f09\" (UID: \"21d92c25-b83a-4f6c-a18d-d2e54eb96f09\") "
Jul 07 11:56:18.793018 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:18.792845    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-istio-data\") pod \"21d92c25-b83a-4f6c-a18d-d2e54eb96f09\" (UID: \"21d92c25-b83a-4f6c-a18d-d2e54eb96f09\") "
Jul 07 11:56:18.793018 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:18.792965    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "21d92c25-b83a-4f6c-a18d-d2e54eb96f09" (UID: "21d92c25-b83a-4f6c-a18d-d2e54eb96f09"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 11:56:18.793018 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:18.792974    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "21d92c25-b83a-4f6c-a18d-d2e54eb96f09" (UID: "21d92c25-b83a-4f6c-a18d-d2e54eb96f09"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:56:18.793305 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:18.793245    2570 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-credential-socket\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:56:18.793305 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:18.793266    2570 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-workload-socket\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:56:18.793305 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:18.793281    2570 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-istiod-ca-cert\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:56:18.793305 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:18.793295    2570 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-workload-certs\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:56:18.793888 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:18.793859    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-istio-data" (OuterVolumeSpecName: "istio-data") pod "21d92c25-b83a-4f6c-a18d-d2e54eb96f09" (UID: "21d92c25-b83a-4f6c-a18d-d2e54eb96f09"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:56:18.795012 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:18.794971    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "21d92c25-b83a-4f6c-a18d-d2e54eb96f09" (UID: "21d92c25-b83a-4f6c-a18d-d2e54eb96f09"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:56:18.795012 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:18.794979    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-kube-api-access-p65p6" (OuterVolumeSpecName: "kube-api-access-p65p6") pod "21d92c25-b83a-4f6c-a18d-d2e54eb96f09" (UID: "21d92c25-b83a-4f6c-a18d-d2e54eb96f09"). InnerVolumeSpecName "kube-api-access-p65p6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:56:18.795161 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:18.795057    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "21d92c25-b83a-4f6c-a18d-d2e54eb96f09" (UID: "21d92c25-b83a-4f6c-a18d-d2e54eb96f09"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 11:56:18.795161 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:18.795079    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-istio-token" (OuterVolumeSpecName: "istio-token") pod "21d92c25-b83a-4f6c-a18d-d2e54eb96f09" (UID: "21d92c25-b83a-4f6c-a18d-d2e54eb96f09"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:56:18.813396 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:18.813362    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8"]
Jul 07 11:56:18.813995 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:18.813967    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21d92c25-b83a-4f6c-a18d-d2e54eb96f09" containerName="istio-proxy"
Jul 07 11:56:18.813995 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:18.813990    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="21d92c25-b83a-4f6c-a18d-d2e54eb96f09" containerName="istio-proxy"
Jul 07 11:56:18.814176 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:18.814072    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="21d92c25-b83a-4f6c-a18d-d2e54eb96f09" containerName="istio-proxy"
Jul 07 11:56:18.817668 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:18.817645    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8"
Jul 07 11:56:18.820846 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:18.820824    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-a4af068a-openshift-default-dockercfg-8lcfm\""
Jul 07 11:56:18.844383 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:18.844351    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8"]
Jul 07 11:56:18.894109 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:18.894076    2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-istio-token\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:56:18.894109 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:18.894107    2570 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-istio-podinfo\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:56:18.894347 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:18.894118    2570 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-istio-data\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:56:18.894347 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:18.894128    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p65p6\" (UniqueName: \"kubernetes.io/projected/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-kube-api-access-p65p6\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:56:18.894347 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:18.894136    2570 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21d92c25-b83a-4f6c-a18d-d2e54eb96f09-istio-envoy\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:56:18.995464 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:18.995423    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e990565-6c54-42d4-ac15-a8778a6474d8-istio-data\") pod \"e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8\" (UID: \"9e990565-6c54-42d4-ac15-a8778a6474d8\") " pod="openshift-ingress/e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8"
Jul 07 11:56:18.995655 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:18.995479    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e990565-6c54-42d4-ac15-a8778a6474d8-credential-socket\") pod \"e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8\" (UID: \"9e990565-6c54-42d4-ac15-a8778a6474d8\") " pod="openshift-ingress/e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8"
Jul 07 11:56:18.995655 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:18.995531    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e990565-6c54-42d4-ac15-a8778a6474d8-workload-socket\") pod \"e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8\" (UID: \"9e990565-6c54-42d4-ac15-a8778a6474d8\") " pod="openshift-ingress/e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8"
Jul 07 11:56:18.995655 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:18.995554    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e990565-6c54-42d4-ac15-a8778a6474d8-workload-certs\") pod \"e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8\" (UID: \"9e990565-6c54-42d4-ac15-a8778a6474d8\") " pod="openshift-ingress/e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8"
Jul 07 11:56:18.995655 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:18.995574    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e990565-6c54-42d4-ac15-a8778a6474d8-istio-token\") pod \"e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8\" (UID: \"9e990565-6c54-42d4-ac15-a8778a6474d8\") " pod="openshift-ingress/e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8"
Jul 07 11:56:18.995655 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:18.995599    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shvsz\" (UniqueName: \"kubernetes.io/projected/9e990565-6c54-42d4-ac15-a8778a6474d8-kube-api-access-shvsz\") pod \"e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8\" (UID: \"9e990565-6c54-42d4-ac15-a8778a6474d8\") " pod="openshift-ingress/e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8"
Jul 07 11:56:18.995829 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:18.995664    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e990565-6c54-42d4-ac15-a8778a6474d8-istio-envoy\") pod \"e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8\" (UID: \"9e990565-6c54-42d4-ac15-a8778a6474d8\") " pod="openshift-ingress/e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8"
Jul 07 11:56:18.995829 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:18.995693    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e990565-6c54-42d4-ac15-a8778a6474d8-istio-podinfo\") pod \"e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8\" (UID: \"9e990565-6c54-42d4-ac15-a8778a6474d8\") " pod="openshift-ingress/e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8"
Jul 07 11:56:18.995829 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:18.995742    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e990565-6c54-42d4-ac15-a8778a6474d8-istiod-ca-cert\") pod \"e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8\" (UID: \"9e990565-6c54-42d4-ac15-a8778a6474d8\") " pod="openshift-ingress/e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8"
Jul 07 11:56:19.096790 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:19.096680    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e990565-6c54-42d4-ac15-a8778a6474d8-workload-socket\") pod \"e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8\" (UID: \"9e990565-6c54-42d4-ac15-a8778a6474d8\") " pod="openshift-ingress/e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8"
Jul 07 11:56:19.096790 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:19.096738    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e990565-6c54-42d4-ac15-a8778a6474d8-workload-certs\") pod \"e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8\" (UID: \"9e990565-6c54-42d4-ac15-a8778a6474d8\") " pod="openshift-ingress/e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8"
Jul 07 11:56:19.096790 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:19.096768    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e990565-6c54-42d4-ac15-a8778a6474d8-istio-token\") pod \"e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8\" (UID: \"9e990565-6c54-42d4-ac15-a8778a6474d8\") " pod="openshift-ingress/e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8"
Jul 07 11:56:19.097075 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:19.096804    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shvsz\" (UniqueName: \"kubernetes.io/projected/9e990565-6c54-42d4-ac15-a8778a6474d8-kube-api-access-shvsz\") pod \"e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8\" (UID: \"9e990565-6c54-42d4-ac15-a8778a6474d8\") " pod="openshift-ingress/e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8"
Jul 07 11:56:19.097075 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:19.096837    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e990565-6c54-42d4-ac15-a8778a6474d8-istio-envoy\") pod \"e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8\" (UID: \"9e990565-6c54-42d4-ac15-a8778a6474d8\") " pod="openshift-ingress/e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8"
Jul 07 11:56:19.097075 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:19.096860    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e990565-6c54-42d4-ac15-a8778a6474d8-istio-podinfo\") pod \"e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8\" (UID: \"9e990565-6c54-42d4-ac15-a8778a6474d8\") " pod="openshift-ingress/e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8"
Jul 07 11:56:19.097075 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:19.096920    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e990565-6c54-42d4-ac15-a8778a6474d8-istiod-ca-cert\") pod \"e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8\" (UID: \"9e990565-6c54-42d4-ac15-a8778a6474d8\") " pod="openshift-ingress/e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8"
Jul 07 11:56:19.097075 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:19.096950    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e990565-6c54-42d4-ac15-a8778a6474d8-istio-data\") pod \"e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8\" (UID: \"9e990565-6c54-42d4-ac15-a8778a6474d8\") " pod="openshift-ingress/e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8"
Jul 07 11:56:19.097075 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:19.096982    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e990565-6c54-42d4-ac15-a8778a6474d8-credential-socket\") pod \"e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8\" (UID: \"9e990565-6c54-42d4-ac15-a8778a6474d8\") " pod="openshift-ingress/e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8"
Jul 07 11:56:19.097410 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:19.097147    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e990565-6c54-42d4-ac15-a8778a6474d8-workload-socket\") pod \"e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8\" (UID: \"9e990565-6c54-42d4-ac15-a8778a6474d8\") " pod="openshift-ingress/e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8"
Jul 07 11:56:19.097410 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:19.097256    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e990565-6c54-42d4-ac15-a8778a6474d8-workload-certs\") pod \"e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8\" (UID: \"9e990565-6c54-42d4-ac15-a8778a6474d8\") " pod="openshift-ingress/e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8"
Jul 07 11:56:19.097410 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:19.097388    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e990565-6c54-42d4-ac15-a8778a6474d8-istio-data\") pod \"e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8\" (UID: \"9e990565-6c54-42d4-ac15-a8778a6474d8\") " pod="openshift-ingress/e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8"
Jul 07 11:56:19.097532 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:19.097429    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e990565-6c54-42d4-ac15-a8778a6474d8-credential-socket\") pod \"e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8\" (UID: \"9e990565-6c54-42d4-ac15-a8778a6474d8\") " pod="openshift-ingress/e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8"
Jul 07 11:56:19.097595 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:19.097578    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e990565-6c54-42d4-ac15-a8778a6474d8-istiod-ca-cert\") pod \"e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8\" (UID: \"9e990565-6c54-42d4-ac15-a8778a6474d8\") " pod="openshift-ingress/e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8"
Jul 07 11:56:19.099234 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:19.099198    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e990565-6c54-42d4-ac15-a8778a6474d8-istio-envoy\") pod \"e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8\" (UID: \"9e990565-6c54-42d4-ac15-a8778a6474d8\") " pod="openshift-ingress/e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8"
Jul 07 11:56:19.099449 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:19.099433    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e990565-6c54-42d4-ac15-a8778a6474d8-istio-podinfo\") pod \"e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8\" (UID: \"9e990565-6c54-42d4-ac15-a8778a6474d8\") " pod="openshift-ingress/e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8"
Jul 07 11:56:19.112857 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:19.112829    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-shvsz\" (UniqueName: \"kubernetes.io/projected/9e990565-6c54-42d4-ac15-a8778a6474d8-kube-api-access-shvsz\") pod \"e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8\" (UID: \"9e990565-6c54-42d4-ac15-a8778a6474d8\") " pod="openshift-ingress/e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8"
Jul 07 11:56:19.113127 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:19.113107    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e990565-6c54-42d4-ac15-a8778a6474d8-istio-token\") pod \"e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8\" (UID: \"9e990565-6c54-42d4-ac15-a8778a6474d8\") " pod="openshift-ingress/e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8"
Jul 07 11:56:19.115551 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:19.115518    2570 generic.go:356] "Generic (PLEG): container finished" podID="21d92c25-b83a-4f6c-a18d-d2e54eb96f09" containerID="b4e4c1abe1fcbc8c7d746ab263841a8e2bf781d32bead53e7d00644deff8763b" exitCode=0
Jul 07 11:56:19.115659 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:19.115572    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx"
Jul 07 11:56:19.115659 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:19.115595    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx" event={"ID":"21d92c25-b83a-4f6c-a18d-d2e54eb96f09","Type":"ContainerDied","Data":"b4e4c1abe1fcbc8c7d746ab263841a8e2bf781d32bead53e7d00644deff8763b"}
Jul 07 11:56:19.115746 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:19.115666    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx" event={"ID":"21d92c25-b83a-4f6c-a18d-d2e54eb96f09","Type":"ContainerDied","Data":"4e47d20b96473b089a2932014e24d1bf144dea1c1b90a38988a6984d93931272"}
Jul 07 11:56:19.115746 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:19.115688    2570 scope.go:117] "RemoveContainer" containerID="b4e4c1abe1fcbc8c7d746ab263841a8e2bf781d32bead53e7d00644deff8763b"
Jul 07 11:56:19.125633 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:19.125621    2570 scope.go:117] "RemoveContainer" containerID="b4e4c1abe1fcbc8c7d746ab263841a8e2bf781d32bead53e7d00644deff8763b"
Jul 07 11:56:19.125887 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:56:19.125869    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4e4c1abe1fcbc8c7d746ab263841a8e2bf781d32bead53e7d00644deff8763b\": container with ID starting with b4e4c1abe1fcbc8c7d746ab263841a8e2bf781d32bead53e7d00644deff8763b not found: ID does not exist" containerID="b4e4c1abe1fcbc8c7d746ab263841a8e2bf781d32bead53e7d00644deff8763b"
Jul 07 11:56:19.125940 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:19.125895    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4e4c1abe1fcbc8c7d746ab263841a8e2bf781d32bead53e7d00644deff8763b"} err="failed to get container status \"b4e4c1abe1fcbc8c7d746ab263841a8e2bf781d32bead53e7d00644deff8763b\": rpc error: code = NotFound desc = could not find container \"b4e4c1abe1fcbc8c7d746ab263841a8e2bf781d32bead53e7d00644deff8763b\": container with ID starting with b4e4c1abe1fcbc8c7d746ab263841a8e2bf781d32bead53e7d00644deff8763b not found: ID does not exist"
Jul 07 11:56:19.128668 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:19.128652    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8"
Jul 07 11:56:19.139769 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:19.139747    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx"]
Jul 07 11:56:19.144129 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:19.144108    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-044dd768-openshift-default-9f8644f5c-rnhhx"]
Jul 07 11:56:19.263993 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:19.263958    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8"]
Jul 07 11:56:19.266087 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:56:19.266058    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e990565_6c54_42d4_ac15_a8778a6474d8.slice/crio-7997b7248ef2098d533248bae6cdedc96f4ba41687dc02ddf435561a5ced1b42 WatchSource:0}: Error finding container 7997b7248ef2098d533248bae6cdedc96f4ba41687dc02ddf435561a5ced1b42: Status 404 returned error can't find the container with id 7997b7248ef2098d533248bae6cdedc96f4ba41687dc02ddf435561a5ced1b42
Jul 07 11:56:19.268136 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:19.268105    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 11:56:19.268254 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:19.268169    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 11:56:19.268254 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:19.268196    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 11:56:19.939077 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:19.939035    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21d92c25-b83a-4f6c-a18d-d2e54eb96f09" path="/var/lib/kubelet/pods/21d92c25-b83a-4f6c-a18d-d2e54eb96f09/volumes"
Jul 07 11:56:20.125429 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:20.125388    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8" event={"ID":"9e990565-6c54-42d4-ac15-a8778a6474d8","Type":"ContainerStarted","Data":"e6df29f1f039e694ac31c9af93da29ed1e8797edbf3cb399ba23ea6704c5cbd0"}
Jul 07 11:56:20.125617 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:20.125458    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8" event={"ID":"9e990565-6c54-42d4-ac15-a8778a6474d8","Type":"ContainerStarted","Data":"7997b7248ef2098d533248bae6cdedc96f4ba41687dc02ddf435561a5ced1b42"}
Jul 07 11:56:20.129811 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:20.129784    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8"
Jul 07 11:56:20.134581 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:20.134556    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8"
Jul 07 11:56:20.157524 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:20.157469    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8" podStartSLOduration=2.157451178 podStartE2EDuration="2.157451178s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:56:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:56:20.155042082 +0000 UTC m=+1768.128682505" watchObservedRunningTime="2026-07-07 11:56:20.157451178 +0000 UTC m=+1768.131091587"
Jul 07 11:56:21.131450 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:21.131417    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8"
Jul 07 11:56:21.132457 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:21.132438    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8"
Jul 07 11:56:31.141355 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:31.141302    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8"]
Jul 07 11:56:31.141877 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:31.141524    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8" podUID="9e990565-6c54-42d4-ac15-a8778a6474d8" containerName="istio-proxy" containerID="cri-o://e6df29f1f039e694ac31c9af93da29ed1e8797edbf3cb399ba23ea6704c5cbd0" gracePeriod=30
Jul 07 11:56:32.106306 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:32.106241    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8"
Jul 07 11:56:32.193933 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:32.193899    2570 generic.go:356] "Generic (PLEG): container finished" podID="9e990565-6c54-42d4-ac15-a8778a6474d8" containerID="e6df29f1f039e694ac31c9af93da29ed1e8797edbf3cb399ba23ea6704c5cbd0" exitCode=0
Jul 07 11:56:32.194372 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:32.193944    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8"
Jul 07 11:56:32.194372 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:32.193959    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8" event={"ID":"9e990565-6c54-42d4-ac15-a8778a6474d8","Type":"ContainerDied","Data":"e6df29f1f039e694ac31c9af93da29ed1e8797edbf3cb399ba23ea6704c5cbd0"}
Jul 07 11:56:32.194372 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:32.194014    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8" event={"ID":"9e990565-6c54-42d4-ac15-a8778a6474d8","Type":"ContainerDied","Data":"7997b7248ef2098d533248bae6cdedc96f4ba41687dc02ddf435561a5ced1b42"}
Jul 07 11:56:32.194372 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:32.194045    2570 scope.go:117] "RemoveContainer" containerID="e6df29f1f039e694ac31c9af93da29ed1e8797edbf3cb399ba23ea6704c5cbd0"
Jul 07 11:56:32.203223 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:32.203192    2570 scope.go:117] "RemoveContainer" containerID="e6df29f1f039e694ac31c9af93da29ed1e8797edbf3cb399ba23ea6704c5cbd0"
Jul 07 11:56:32.203530 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:56:32.203507    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6df29f1f039e694ac31c9af93da29ed1e8797edbf3cb399ba23ea6704c5cbd0\": container with ID starting with e6df29f1f039e694ac31c9af93da29ed1e8797edbf3cb399ba23ea6704c5cbd0 not found: ID does not exist" containerID="e6df29f1f039e694ac31c9af93da29ed1e8797edbf3cb399ba23ea6704c5cbd0"
Jul 07 11:56:32.203606 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:32.203540    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6df29f1f039e694ac31c9af93da29ed1e8797edbf3cb399ba23ea6704c5cbd0"} err="failed to get container status \"e6df29f1f039e694ac31c9af93da29ed1e8797edbf3cb399ba23ea6704c5cbd0\": rpc error: code = NotFound desc = could not find container \"e6df29f1f039e694ac31c9af93da29ed1e8797edbf3cb399ba23ea6704c5cbd0\": container with ID starting with e6df29f1f039e694ac31c9af93da29ed1e8797edbf3cb399ba23ea6704c5cbd0 not found: ID does not exist"
Jul 07 11:56:32.208824 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:32.208804    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e990565-6c54-42d4-ac15-a8778a6474d8-istio-data\") pod \"9e990565-6c54-42d4-ac15-a8778a6474d8\" (UID: \"9e990565-6c54-42d4-ac15-a8778a6474d8\") "
Jul 07 11:56:32.208903 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:32.208835    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-shvsz\" (UniqueName: \"kubernetes.io/projected/9e990565-6c54-42d4-ac15-a8778a6474d8-kube-api-access-shvsz\") pod \"9e990565-6c54-42d4-ac15-a8778a6474d8\" (UID: \"9e990565-6c54-42d4-ac15-a8778a6474d8\") "
Jul 07 11:56:32.208903 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:32.208859    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e990565-6c54-42d4-ac15-a8778a6474d8-workload-socket\") pod \"9e990565-6c54-42d4-ac15-a8778a6474d8\" (UID: \"9e990565-6c54-42d4-ac15-a8778a6474d8\") "
Jul 07 11:56:32.208903 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:32.208886    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e990565-6c54-42d4-ac15-a8778a6474d8-credential-socket\") pod \"9e990565-6c54-42d4-ac15-a8778a6474d8\" (UID: \"9e990565-6c54-42d4-ac15-a8778a6474d8\") "
Jul 07 11:56:32.209061 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:32.208915    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e990565-6c54-42d4-ac15-a8778a6474d8-workload-certs\") pod \"9e990565-6c54-42d4-ac15-a8778a6474d8\" (UID: \"9e990565-6c54-42d4-ac15-a8778a6474d8\") "
Jul 07 11:56:32.209061 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:32.208952    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e990565-6c54-42d4-ac15-a8778a6474d8-istio-token\") pod \"9e990565-6c54-42d4-ac15-a8778a6474d8\" (UID: \"9e990565-6c54-42d4-ac15-a8778a6474d8\") "
Jul 07 11:56:32.209061 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:32.208969    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e990565-6c54-42d4-ac15-a8778a6474d8-istio-podinfo\") pod \"9e990565-6c54-42d4-ac15-a8778a6474d8\" (UID: \"9e990565-6c54-42d4-ac15-a8778a6474d8\") "
Jul 07 11:56:32.209061 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:32.209003    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e990565-6c54-42d4-ac15-a8778a6474d8-istio-envoy\") pod \"9e990565-6c54-42d4-ac15-a8778a6474d8\" (UID: \"9e990565-6c54-42d4-ac15-a8778a6474d8\") "
Jul 07 11:56:32.209292 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:32.209078    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e990565-6c54-42d4-ac15-a8778a6474d8-istiod-ca-cert\") pod \"9e990565-6c54-42d4-ac15-a8778a6474d8\" (UID: \"9e990565-6c54-42d4-ac15-a8778a6474d8\") "
Jul 07 11:56:32.209292 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:32.209095    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e990565-6c54-42d4-ac15-a8778a6474d8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9e990565-6c54-42d4-ac15-a8778a6474d8" (UID: "9e990565-6c54-42d4-ac15-a8778a6474d8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:56:32.209401 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:32.209323    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e990565-6c54-42d4-ac15-a8778a6474d8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9e990565-6c54-42d4-ac15-a8778a6474d8" (UID: "9e990565-6c54-42d4-ac15-a8778a6474d8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:56:32.209401 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:32.209379    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e990565-6c54-42d4-ac15-a8778a6474d8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9e990565-6c54-42d4-ac15-a8778a6474d8" (UID: "9e990565-6c54-42d4-ac15-a8778a6474d8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:56:32.209401 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:32.209394    2570 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e990565-6c54-42d4-ac15-a8778a6474d8-workload-socket\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:56:32.209551 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:32.209416    2570 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e990565-6c54-42d4-ac15-a8778a6474d8-credential-socket\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:56:32.209831 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:32.209806    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e990565-6c54-42d4-ac15-a8778a6474d8-istio-data" (OuterVolumeSpecName: "istio-data") pod "9e990565-6c54-42d4-ac15-a8778a6474d8" (UID: "9e990565-6c54-42d4-ac15-a8778a6474d8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:56:32.209831 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:32.209820    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e990565-6c54-42d4-ac15-a8778a6474d8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9e990565-6c54-42d4-ac15-a8778a6474d8" (UID: "9e990565-6c54-42d4-ac15-a8778a6474d8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 11:56:32.211105 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:32.211077    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e990565-6c54-42d4-ac15-a8778a6474d8-kube-api-access-shvsz" (OuterVolumeSpecName: "kube-api-access-shvsz") pod "9e990565-6c54-42d4-ac15-a8778a6474d8" (UID: "9e990565-6c54-42d4-ac15-a8778a6474d8"). InnerVolumeSpecName "kube-api-access-shvsz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:56:32.211285 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:32.211261    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e990565-6c54-42d4-ac15-a8778a6474d8-istio-token" (OuterVolumeSpecName: "istio-token") pod "9e990565-6c54-42d4-ac15-a8778a6474d8" (UID: "9e990565-6c54-42d4-ac15-a8778a6474d8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:56:32.211532 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:32.211509    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9e990565-6c54-42d4-ac15-a8778a6474d8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9e990565-6c54-42d4-ac15-a8778a6474d8" (UID: "9e990565-6c54-42d4-ac15-a8778a6474d8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 11:56:32.211603 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:32.211585    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e990565-6c54-42d4-ac15-a8778a6474d8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9e990565-6c54-42d4-ac15-a8778a6474d8" (UID: "9e990565-6c54-42d4-ac15-a8778a6474d8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:56:32.310670 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:32.310631    2570 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e990565-6c54-42d4-ac15-a8778a6474d8-istio-data\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:56:32.310670 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:32.310662    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-shvsz\" (UniqueName: \"kubernetes.io/projected/9e990565-6c54-42d4-ac15-a8778a6474d8-kube-api-access-shvsz\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:56:32.310670 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:32.310673    2570 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e990565-6c54-42d4-ac15-a8778a6474d8-workload-certs\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:56:32.310912 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:32.310682    2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e990565-6c54-42d4-ac15-a8778a6474d8-istio-token\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:56:32.310912 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:32.310690    2570 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e990565-6c54-42d4-ac15-a8778a6474d8-istio-podinfo\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:56:32.310912 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:32.310698    2570 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e990565-6c54-42d4-ac15-a8778a6474d8-istio-envoy\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:56:32.310912 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:32.310705    2570 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e990565-6c54-42d4-ac15-a8778a6474d8-istiod-ca-cert\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:56:32.519505 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:32.519462    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8"]
Jul 07 11:56:32.521702 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:32.521674    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-a4af068a-openshift-default-85858d8d5-s7wb8"]
Jul 07 11:56:33.938799 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:33.938769    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e990565-6c54-42d4-ac15-a8778a6474d8" path="/var/lib/kubelet/pods/9e990565-6c54-42d4-ac15-a8778a6474d8/volumes"
Jul 07 11:56:37.261435 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:37.261399    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r"]
Jul 07 11:56:37.261939 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:37.261753    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e990565-6c54-42d4-ac15-a8778a6474d8" containerName="istio-proxy"
Jul 07 11:56:37.261939 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:37.261764    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e990565-6c54-42d4-ac15-a8778a6474d8" containerName="istio-proxy"
Jul 07 11:56:37.261939 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:37.261832    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e990565-6c54-42d4-ac15-a8778a6474d8" containerName="istio-proxy"
Jul 07 11:56:37.266150 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:37.266129    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r"
Jul 07 11:56:37.269019 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:37.268995    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-9131654a-openshift-default-dockercfg-t7jpq\""
Jul 07 11:56:37.283417 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:37.283393    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r"]
Jul 07 11:56:37.358685 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:37.358643    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5pt8\" (UniqueName: \"kubernetes.io/projected/b2d99f46-6a46-4ccc-9644-de614442b38d-kube-api-access-q5pt8\") pod \"e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r\" (UID: \"b2d99f46-6a46-4ccc-9644-de614442b38d\") " pod="openshift-ingress/e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r"
Jul 07 11:56:37.358912 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:37.358697    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2d99f46-6a46-4ccc-9644-de614442b38d-credential-socket\") pod \"e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r\" (UID: \"b2d99f46-6a46-4ccc-9644-de614442b38d\") " pod="openshift-ingress/e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r"
Jul 07 11:56:37.358912 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:37.358725    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2d99f46-6a46-4ccc-9644-de614442b38d-istio-podinfo\") pod \"e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r\" (UID: \"b2d99f46-6a46-4ccc-9644-de614442b38d\") " pod="openshift-ingress/e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r"
Jul 07 11:56:37.358912 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:37.358869    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2d99f46-6a46-4ccc-9644-de614442b38d-workload-socket\") pod \"e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r\" (UID: \"b2d99f46-6a46-4ccc-9644-de614442b38d\") " pod="openshift-ingress/e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r"
Jul 07 11:56:37.359077 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:37.358909    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2d99f46-6a46-4ccc-9644-de614442b38d-istio-data\") pod \"e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r\" (UID: \"b2d99f46-6a46-4ccc-9644-de614442b38d\") " pod="openshift-ingress/e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r"
Jul 07 11:56:37.359077 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:37.358941    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2d99f46-6a46-4ccc-9644-de614442b38d-workload-certs\") pod \"e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r\" (UID: \"b2d99f46-6a46-4ccc-9644-de614442b38d\") " pod="openshift-ingress/e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r"
Jul 07 11:56:37.359077 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:37.358966    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2d99f46-6a46-4ccc-9644-de614442b38d-istio-token\") pod \"e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r\" (UID: \"b2d99f46-6a46-4ccc-9644-de614442b38d\") " pod="openshift-ingress/e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r"
Jul 07 11:56:37.359077 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:37.358992    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2d99f46-6a46-4ccc-9644-de614442b38d-istiod-ca-cert\") pod \"e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r\" (UID: \"b2d99f46-6a46-4ccc-9644-de614442b38d\") " pod="openshift-ingress/e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r"
Jul 07 11:56:37.359482 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:37.359108    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2d99f46-6a46-4ccc-9644-de614442b38d-istio-envoy\") pod \"e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r\" (UID: \"b2d99f46-6a46-4ccc-9644-de614442b38d\") " pod="openshift-ingress/e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r"
Jul 07 11:56:37.459636 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:37.459576    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2d99f46-6a46-4ccc-9644-de614442b38d-workload-socket\") pod \"e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r\" (UID: \"b2d99f46-6a46-4ccc-9644-de614442b38d\") " pod="openshift-ingress/e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r"
Jul 07 11:56:37.459636 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:37.459633    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2d99f46-6a46-4ccc-9644-de614442b38d-istio-data\") pod \"e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r\" (UID: \"b2d99f46-6a46-4ccc-9644-de614442b38d\") " pod="openshift-ingress/e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r"
Jul 07 11:56:37.459937 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:37.459793    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2d99f46-6a46-4ccc-9644-de614442b38d-workload-certs\") pod \"e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r\" (UID: \"b2d99f46-6a46-4ccc-9644-de614442b38d\") " pod="openshift-ingress/e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r"
Jul 07 11:56:37.459937 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:37.459811    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2d99f46-6a46-4ccc-9644-de614442b38d-istio-token\") pod \"e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r\" (UID: \"b2d99f46-6a46-4ccc-9644-de614442b38d\") " pod="openshift-ingress/e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r"
Jul 07 11:56:37.459937 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:37.459828    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2d99f46-6a46-4ccc-9644-de614442b38d-istiod-ca-cert\") pod \"e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r\" (UID: \"b2d99f46-6a46-4ccc-9644-de614442b38d\") " pod="openshift-ingress/e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r"
Jul 07 11:56:37.459937 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:37.459902    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2d99f46-6a46-4ccc-9644-de614442b38d-istio-envoy\") pod \"e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r\" (UID: \"b2d99f46-6a46-4ccc-9644-de614442b38d\") " pod="openshift-ingress/e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r"
Jul 07 11:56:37.460182 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:37.460084    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5pt8\" (UniqueName: \"kubernetes.io/projected/b2d99f46-6a46-4ccc-9644-de614442b38d-kube-api-access-q5pt8\") pod \"e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r\" (UID: \"b2d99f46-6a46-4ccc-9644-de614442b38d\") " pod="openshift-ingress/e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r"
Jul 07 11:56:37.460182 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:37.460141    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2d99f46-6a46-4ccc-9644-de614442b38d-credential-socket\") pod \"e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r\" (UID: \"b2d99f46-6a46-4ccc-9644-de614442b38d\") " pod="openshift-ingress/e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r"
Jul 07 11:56:37.460182 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:37.460166    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2d99f46-6a46-4ccc-9644-de614442b38d-istio-podinfo\") pod \"e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r\" (UID: \"b2d99f46-6a46-4ccc-9644-de614442b38d\") " pod="openshift-ingress/e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r"
Jul 07 11:56:37.460425 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:37.460092    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2d99f46-6a46-4ccc-9644-de614442b38d-workload-socket\") pod \"e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r\" (UID: \"b2d99f46-6a46-4ccc-9644-de614442b38d\") " pod="openshift-ingress/e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r"
Jul 07 11:56:37.460425 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:37.460183    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2d99f46-6a46-4ccc-9644-de614442b38d-workload-certs\") pod \"e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r\" (UID: \"b2d99f46-6a46-4ccc-9644-de614442b38d\") " pod="openshift-ingress/e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r"
Jul 07 11:56:37.460425 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:37.460335    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2d99f46-6a46-4ccc-9644-de614442b38d-istio-data\") pod \"e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r\" (UID: \"b2d99f46-6a46-4ccc-9644-de614442b38d\") " pod="openshift-ingress/e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r"
Jul 07 11:56:37.460538 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:37.460499    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2d99f46-6a46-4ccc-9644-de614442b38d-credential-socket\") pod \"e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r\" (UID: \"b2d99f46-6a46-4ccc-9644-de614442b38d\") " pod="openshift-ingress/e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r"
Jul 07 11:56:37.460641 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:37.460622    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2d99f46-6a46-4ccc-9644-de614442b38d-istiod-ca-cert\") pod \"e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r\" (UID: \"b2d99f46-6a46-4ccc-9644-de614442b38d\") " pod="openshift-ingress/e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r"
Jul 07 11:56:37.462359 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:37.462336    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2d99f46-6a46-4ccc-9644-de614442b38d-istio-envoy\") pod \"e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r\" (UID: \"b2d99f46-6a46-4ccc-9644-de614442b38d\") " pod="openshift-ingress/e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r"
Jul 07 11:56:37.462599 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:37.462581    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2d99f46-6a46-4ccc-9644-de614442b38d-istio-podinfo\") pod \"e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r\" (UID: \"b2d99f46-6a46-4ccc-9644-de614442b38d\") " pod="openshift-ingress/e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r"
Jul 07 11:56:37.469111 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:37.469088    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5pt8\" (UniqueName: \"kubernetes.io/projected/b2d99f46-6a46-4ccc-9644-de614442b38d-kube-api-access-q5pt8\") pod \"e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r\" (UID: \"b2d99f46-6a46-4ccc-9644-de614442b38d\") " pod="openshift-ingress/e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r"
Jul 07 11:56:37.469387 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:37.469367    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2d99f46-6a46-4ccc-9644-de614442b38d-istio-token\") pod \"e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r\" (UID: \"b2d99f46-6a46-4ccc-9644-de614442b38d\") " pod="openshift-ingress/e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r"
Jul 07 11:56:37.578456 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:37.578355    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r"
Jul 07 11:56:37.724811 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:37.724777    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r"]
Jul 07 11:56:37.725586 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:56:37.725560    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2d99f46_6a46_4ccc_9644_de614442b38d.slice/crio-8e1f9ed585a2a9caa671b5832154f8619fa78440d648be447adcde8c451e3e64 WatchSource:0}: Error finding container 8e1f9ed585a2a9caa671b5832154f8619fa78440d648be447adcde8c451e3e64: Status 404 returned error can't find the container with id 8e1f9ed585a2a9caa671b5832154f8619fa78440d648be447adcde8c451e3e64
Jul 07 11:56:37.727907 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:37.727867    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 11:56:37.727970 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:37.727958    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 11:56:37.728022 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:37.728005    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 11:56:38.227901 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:38.227863    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r" event={"ID":"b2d99f46-6a46-4ccc-9644-de614442b38d","Type":"ContainerStarted","Data":"1a43fcbd0c7e320bc1fbfba33bdefcbe3f06aaace08c2dbd754fe0fea6bb62fe"}
Jul 07 11:56:38.228133 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:38.227921    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r" event={"ID":"b2d99f46-6a46-4ccc-9644-de614442b38d","Type":"ContainerStarted","Data":"8e1f9ed585a2a9caa671b5832154f8619fa78440d648be447adcde8c451e3e64"}
Jul 07 11:56:38.249189 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:38.249136    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r" podStartSLOduration=1.249120158 podStartE2EDuration="1.249120158s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:56:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:56:38.247393034 +0000 UTC m=+1786.221033444" watchObservedRunningTime="2026-07-07 11:56:38.249120158 +0000 UTC m=+1786.222760604"
Jul 07 11:56:38.579422 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:38.579323    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r"
Jul 07 11:56:38.584260 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:38.584222    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r"
Jul 07 11:56:39.232985 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:39.232956    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r"
Jul 07 11:56:39.234020 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:39.233997    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r"
Jul 07 11:56:42.968186 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:42.968146    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r"]
Jul 07 11:56:42.968665 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:42.968375    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r" podUID="b2d99f46-6a46-4ccc-9644-de614442b38d" containerName="istio-proxy" containerID="cri-o://1a43fcbd0c7e320bc1fbfba33bdefcbe3f06aaace08c2dbd754fe0fea6bb62fe" gracePeriod=30
Jul 07 11:56:49.022632 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:49.022575    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r"
Jul 07 11:56:49.084600 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:49.084547    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2d99f46-6a46-4ccc-9644-de614442b38d-workload-socket\") pod \"b2d99f46-6a46-4ccc-9644-de614442b38d\" (UID: \"b2d99f46-6a46-4ccc-9644-de614442b38d\") "
Jul 07 11:56:49.084600 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:49.084605    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2d99f46-6a46-4ccc-9644-de614442b38d-istio-token\") pod \"b2d99f46-6a46-4ccc-9644-de614442b38d\" (UID: \"b2d99f46-6a46-4ccc-9644-de614442b38d\") "
Jul 07 11:56:49.084872 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:49.084690    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2d99f46-6a46-4ccc-9644-de614442b38d-credential-socket\") pod \"b2d99f46-6a46-4ccc-9644-de614442b38d\" (UID: \"b2d99f46-6a46-4ccc-9644-de614442b38d\") "
Jul 07 11:56:49.084872 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:49.084712    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q5pt8\" (UniqueName: \"kubernetes.io/projected/b2d99f46-6a46-4ccc-9644-de614442b38d-kube-api-access-q5pt8\") pod \"b2d99f46-6a46-4ccc-9644-de614442b38d\" (UID: \"b2d99f46-6a46-4ccc-9644-de614442b38d\") "
Jul 07 11:56:49.084872 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:49.084740    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2d99f46-6a46-4ccc-9644-de614442b38d-istio-podinfo\") pod \"b2d99f46-6a46-4ccc-9644-de614442b38d\" (UID: \"b2d99f46-6a46-4ccc-9644-de614442b38d\") "
Jul 07 11:56:49.084872 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:49.084759    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2d99f46-6a46-4ccc-9644-de614442b38d-istio-data\") pod \"b2d99f46-6a46-4ccc-9644-de614442b38d\" (UID: \"b2d99f46-6a46-4ccc-9644-de614442b38d\") "
Jul 07 11:56:49.084872 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:49.084775    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2d99f46-6a46-4ccc-9644-de614442b38d-workload-certs\") pod \"b2d99f46-6a46-4ccc-9644-de614442b38d\" (UID: \"b2d99f46-6a46-4ccc-9644-de614442b38d\") "
Jul 07 11:56:49.084872 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:49.084795    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2d99f46-6a46-4ccc-9644-de614442b38d-istiod-ca-cert\") pod \"b2d99f46-6a46-4ccc-9644-de614442b38d\" (UID: \"b2d99f46-6a46-4ccc-9644-de614442b38d\") "
Jul 07 11:56:49.084872 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:49.084828    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2d99f46-6a46-4ccc-9644-de614442b38d-istio-envoy\") pod \"b2d99f46-6a46-4ccc-9644-de614442b38d\" (UID: \"b2d99f46-6a46-4ccc-9644-de614442b38d\") "
Jul 07 11:56:49.084872 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:49.084847    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2d99f46-6a46-4ccc-9644-de614442b38d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b2d99f46-6a46-4ccc-9644-de614442b38d" (UID: "b2d99f46-6a46-4ccc-9644-de614442b38d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:56:49.085405 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:49.085087    2570 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2d99f46-6a46-4ccc-9644-de614442b38d-workload-socket\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:56:49.085405 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:49.085139    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2d99f46-6a46-4ccc-9644-de614442b38d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b2d99f46-6a46-4ccc-9644-de614442b38d" (UID: "b2d99f46-6a46-4ccc-9644-de614442b38d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:56:49.085405 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:49.085260    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2d99f46-6a46-4ccc-9644-de614442b38d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b2d99f46-6a46-4ccc-9644-de614442b38d" (UID: "b2d99f46-6a46-4ccc-9644-de614442b38d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:56:49.085567 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:49.085405    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2d99f46-6a46-4ccc-9644-de614442b38d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b2d99f46-6a46-4ccc-9644-de614442b38d" (UID: "b2d99f46-6a46-4ccc-9644-de614442b38d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 11:56:49.085992 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:49.085958    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2d99f46-6a46-4ccc-9644-de614442b38d-istio-data" (OuterVolumeSpecName: "istio-data") pod "b2d99f46-6a46-4ccc-9644-de614442b38d" (UID: "b2d99f46-6a46-4ccc-9644-de614442b38d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:56:49.087033 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:49.087005    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2d99f46-6a46-4ccc-9644-de614442b38d-istio-token" (OuterVolumeSpecName: "istio-token") pod "b2d99f46-6a46-4ccc-9644-de614442b38d" (UID: "b2d99f46-6a46-4ccc-9644-de614442b38d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:56:49.087033 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:49.087027    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b2d99f46-6a46-4ccc-9644-de614442b38d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b2d99f46-6a46-4ccc-9644-de614442b38d" (UID: "b2d99f46-6a46-4ccc-9644-de614442b38d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 11:56:49.087169 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:49.087077    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2d99f46-6a46-4ccc-9644-de614442b38d-kube-api-access-q5pt8" (OuterVolumeSpecName: "kube-api-access-q5pt8") pod "b2d99f46-6a46-4ccc-9644-de614442b38d" (UID: "b2d99f46-6a46-4ccc-9644-de614442b38d"). InnerVolumeSpecName "kube-api-access-q5pt8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:56:49.087169 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:49.087128    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2d99f46-6a46-4ccc-9644-de614442b38d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b2d99f46-6a46-4ccc-9644-de614442b38d" (UID: "b2d99f46-6a46-4ccc-9644-de614442b38d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:56:49.185888 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:49.185857    2570 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2d99f46-6a46-4ccc-9644-de614442b38d-istio-envoy\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:56:49.185888 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:49.185885    2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2d99f46-6a46-4ccc-9644-de614442b38d-istio-token\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:56:49.186093 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:49.185895    2570 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2d99f46-6a46-4ccc-9644-de614442b38d-credential-socket\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:56:49.186093 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:49.185904    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q5pt8\" (UniqueName: \"kubernetes.io/projected/b2d99f46-6a46-4ccc-9644-de614442b38d-kube-api-access-q5pt8\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:56:49.186093 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:49.185915    2570 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2d99f46-6a46-4ccc-9644-de614442b38d-istio-podinfo\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:56:49.186093 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:49.185924    2570 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2d99f46-6a46-4ccc-9644-de614442b38d-istio-data\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:56:49.186093 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:49.185932    2570 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2d99f46-6a46-4ccc-9644-de614442b38d-workload-certs\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:56:49.186093 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:49.185940    2570 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2d99f46-6a46-4ccc-9644-de614442b38d-istiod-ca-cert\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:56:49.290732 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:49.290694    2570 generic.go:356] "Generic (PLEG): container finished" podID="b2d99f46-6a46-4ccc-9644-de614442b38d" containerID="1a43fcbd0c7e320bc1fbfba33bdefcbe3f06aaace08c2dbd754fe0fea6bb62fe" exitCode=0
Jul 07 11:56:49.290937 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:49.290747    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r"
Jul 07 11:56:49.290937 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:49.290778    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r" event={"ID":"b2d99f46-6a46-4ccc-9644-de614442b38d","Type":"ContainerDied","Data":"1a43fcbd0c7e320bc1fbfba33bdefcbe3f06aaace08c2dbd754fe0fea6bb62fe"}
Jul 07 11:56:49.290937 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:49.290859    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r" event={"ID":"b2d99f46-6a46-4ccc-9644-de614442b38d","Type":"ContainerDied","Data":"8e1f9ed585a2a9caa671b5832154f8619fa78440d648be447adcde8c451e3e64"}
Jul 07 11:56:49.290937 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:49.290882    2570 scope.go:117] "RemoveContainer" containerID="1a43fcbd0c7e320bc1fbfba33bdefcbe3f06aaace08c2dbd754fe0fea6bb62fe"
Jul 07 11:56:49.300832 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:49.300816    2570 scope.go:117] "RemoveContainer" containerID="1a43fcbd0c7e320bc1fbfba33bdefcbe3f06aaace08c2dbd754fe0fea6bb62fe"
Jul 07 11:56:49.301089 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:56:49.301069    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a43fcbd0c7e320bc1fbfba33bdefcbe3f06aaace08c2dbd754fe0fea6bb62fe\": container with ID starting with 1a43fcbd0c7e320bc1fbfba33bdefcbe3f06aaace08c2dbd754fe0fea6bb62fe not found: ID does not exist" containerID="1a43fcbd0c7e320bc1fbfba33bdefcbe3f06aaace08c2dbd754fe0fea6bb62fe"
Jul 07 11:56:49.301165 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:49.301102    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a43fcbd0c7e320bc1fbfba33bdefcbe3f06aaace08c2dbd754fe0fea6bb62fe"} err="failed to get container status \"1a43fcbd0c7e320bc1fbfba33bdefcbe3f06aaace08c2dbd754fe0fea6bb62fe\": rpc error: code = NotFound desc = could not find container \"1a43fcbd0c7e320bc1fbfba33bdefcbe3f06aaace08c2dbd754fe0fea6bb62fe\": container with ID starting with 1a43fcbd0c7e320bc1fbfba33bdefcbe3f06aaace08c2dbd754fe0fea6bb62fe not found: ID does not exist"
Jul 07 11:56:49.318957 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:49.318921    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r"]
Jul 07 11:56:49.330653 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:49.330631    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-9131654a-openshift-default-6b79cd4c99-gws8r"]
Jul 07 11:56:49.938410 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:49.938376    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2d99f46-6a46-4ccc-9644-de614442b38d" path="/var/lib/kubelet/pods/b2d99f46-6a46-4ccc-9644-de614442b38d/volumes"
Jul 07 11:56:54.001611 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:54.001581    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ktbpw_faf6f343-6515-4b2c-a34e-286cd2d9ec7a/console-operator/1.log"
Jul 07 11:56:54.009555 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:56:54.009521    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ktbpw_faf6f343-6515-4b2c-a34e-286cd2d9ec7a/console-operator/1.log"
Jul 07 11:58:24.592510 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:24.592474    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch"]
Jul 07 11:58:24.593063 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:24.592966    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2d99f46-6a46-4ccc-9644-de614442b38d" containerName="istio-proxy"
Jul 07 11:58:24.593063 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:24.592988    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2d99f46-6a46-4ccc-9644-de614442b38d" containerName="istio-proxy"
Jul 07 11:58:24.593253 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:24.593084    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2d99f46-6a46-4ccc-9644-de614442b38d" containerName="istio-proxy"
Jul 07 11:58:24.596020 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:24.596003    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch"
Jul 07 11:58:24.598885 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:24.598861    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-5e15070c-openshift-default-dockercfg-ngrgm\""
Jul 07 11:58:24.616314 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:24.616288    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch"]
Jul 07 11:58:24.745389 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:24.745348    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9ca4afb-55ce-40b2-9205-dfa800f39865-workload-socket\") pod \"e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch\" (UID: \"c9ca4afb-55ce-40b2-9205-dfa800f39865\") " pod="openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch"
Jul 07 11:58:24.745587 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:24.745410    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9ca4afb-55ce-40b2-9205-dfa800f39865-istio-podinfo\") pod \"e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch\" (UID: \"c9ca4afb-55ce-40b2-9205-dfa800f39865\") " pod="openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch"
Jul 07 11:58:24.745587 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:24.745440    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9ca4afb-55ce-40b2-9205-dfa800f39865-workload-certs\") pod \"e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch\" (UID: \"c9ca4afb-55ce-40b2-9205-dfa800f39865\") " pod="openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch"
Jul 07 11:58:24.745587 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:24.745466    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9ca4afb-55ce-40b2-9205-dfa800f39865-istio-token\") pod \"e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch\" (UID: \"c9ca4afb-55ce-40b2-9205-dfa800f39865\") " pod="openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch"
Jul 07 11:58:24.745719 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:24.745581    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9ca4afb-55ce-40b2-9205-dfa800f39865-istio-envoy\") pod \"e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch\" (UID: \"c9ca4afb-55ce-40b2-9205-dfa800f39865\") " pod="openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch"
Jul 07 11:58:24.745719 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:24.745615    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9ca4afb-55ce-40b2-9205-dfa800f39865-istiod-ca-cert\") pod \"e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch\" (UID: \"c9ca4afb-55ce-40b2-9205-dfa800f39865\") " pod="openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch"
Jul 07 11:58:24.745719 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:24.745634    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9ca4afb-55ce-40b2-9205-dfa800f39865-credential-socket\") pod \"e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch\" (UID: \"c9ca4afb-55ce-40b2-9205-dfa800f39865\") " pod="openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch"
Jul 07 11:58:24.745719 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:24.745676    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9ca4afb-55ce-40b2-9205-dfa800f39865-istio-data\") pod \"e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch\" (UID: \"c9ca4afb-55ce-40b2-9205-dfa800f39865\") " pod="openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch"
Jul 07 11:58:24.745719 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:24.745711    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgffw\" (UniqueName: \"kubernetes.io/projected/c9ca4afb-55ce-40b2-9205-dfa800f39865-kube-api-access-dgffw\") pod \"e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch\" (UID: \"c9ca4afb-55ce-40b2-9205-dfa800f39865\") " pod="openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch"
Jul 07 11:58:24.846859 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:24.846771    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9ca4afb-55ce-40b2-9205-dfa800f39865-workload-socket\") pod \"e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch\" (UID: \"c9ca4afb-55ce-40b2-9205-dfa800f39865\") " pod="openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch"
Jul 07 11:58:24.846859 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:24.846832    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9ca4afb-55ce-40b2-9205-dfa800f39865-istio-podinfo\") pod \"e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch\" (UID: \"c9ca4afb-55ce-40b2-9205-dfa800f39865\") " pod="openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch"
Jul 07 11:58:24.847087 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:24.846867    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9ca4afb-55ce-40b2-9205-dfa800f39865-workload-certs\") pod \"e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch\" (UID: \"c9ca4afb-55ce-40b2-9205-dfa800f39865\") " pod="openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch"
Jul 07 11:58:24.847087 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:24.846895    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9ca4afb-55ce-40b2-9205-dfa800f39865-istio-token\") pod \"e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch\" (UID: \"c9ca4afb-55ce-40b2-9205-dfa800f39865\") " pod="openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch"
Jul 07 11:58:24.847087 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:24.846965    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9ca4afb-55ce-40b2-9205-dfa800f39865-istio-envoy\") pod \"e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch\" (UID: \"c9ca4afb-55ce-40b2-9205-dfa800f39865\") " pod="openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch"
Jul 07 11:58:24.847087 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:24.846997    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9ca4afb-55ce-40b2-9205-dfa800f39865-istiod-ca-cert\") pod \"e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch\" (UID: \"c9ca4afb-55ce-40b2-9205-dfa800f39865\") " pod="openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch"
Jul 07 11:58:24.847087 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:24.847026    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9ca4afb-55ce-40b2-9205-dfa800f39865-credential-socket\") pod \"e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch\" (UID: \"c9ca4afb-55ce-40b2-9205-dfa800f39865\") " pod="openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch"
Jul 07 11:58:24.847556 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:24.847512    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9ca4afb-55ce-40b2-9205-dfa800f39865-istio-data\") pod \"e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch\" (UID: \"c9ca4afb-55ce-40b2-9205-dfa800f39865\") " pod="openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch"
Jul 07 11:58:24.847639 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:24.847585    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgffw\" (UniqueName: \"kubernetes.io/projected/c9ca4afb-55ce-40b2-9205-dfa800f39865-kube-api-access-dgffw\") pod \"e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch\" (UID: \"c9ca4afb-55ce-40b2-9205-dfa800f39865\") " pod="openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch"
Jul 07 11:58:24.847639 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:24.847588    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9ca4afb-55ce-40b2-9205-dfa800f39865-workload-certs\") pod \"e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch\" (UID: \"c9ca4afb-55ce-40b2-9205-dfa800f39865\") " pod="openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch"
Jul 07 11:58:24.847874 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:24.847846    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9ca4afb-55ce-40b2-9205-dfa800f39865-istio-data\") pod \"e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch\" (UID: \"c9ca4afb-55ce-40b2-9205-dfa800f39865\") " pod="openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch"
Jul 07 11:58:24.847940 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:24.847904    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9ca4afb-55ce-40b2-9205-dfa800f39865-workload-socket\") pod \"e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch\" (UID: \"c9ca4afb-55ce-40b2-9205-dfa800f39865\") " pod="openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch"
Jul 07 11:58:24.848097 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:24.848072    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9ca4afb-55ce-40b2-9205-dfa800f39865-credential-socket\") pod \"e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch\" (UID: \"c9ca4afb-55ce-40b2-9205-dfa800f39865\") " pod="openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch"
Jul 07 11:58:24.848634 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:24.848544    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9ca4afb-55ce-40b2-9205-dfa800f39865-istiod-ca-cert\") pod \"e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch\" (UID: \"c9ca4afb-55ce-40b2-9205-dfa800f39865\") " pod="openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch"
Jul 07 11:58:24.853906 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:24.853879    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9ca4afb-55ce-40b2-9205-dfa800f39865-istio-envoy\") pod \"e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch\" (UID: \"c9ca4afb-55ce-40b2-9205-dfa800f39865\") " pod="openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch"
Jul 07 11:58:24.854248 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:24.854198    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9ca4afb-55ce-40b2-9205-dfa800f39865-istio-podinfo\") pod \"e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch\" (UID: \"c9ca4afb-55ce-40b2-9205-dfa800f39865\") " pod="openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch"
Jul 07 11:58:24.856592 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:24.856521    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9ca4afb-55ce-40b2-9205-dfa800f39865-istio-token\") pod \"e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch\" (UID: \"c9ca4afb-55ce-40b2-9205-dfa800f39865\") " pod="openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch"
Jul 07 11:58:24.857430 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:24.857410    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgffw\" (UniqueName: \"kubernetes.io/projected/c9ca4afb-55ce-40b2-9205-dfa800f39865-kube-api-access-dgffw\") pod \"e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch\" (UID: \"c9ca4afb-55ce-40b2-9205-dfa800f39865\") " pod="openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch"
Jul 07 11:58:24.906543 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:24.906504    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch"
Jul 07 11:58:25.049864 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:25.049833    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch"]
Jul 07 11:58:25.051344 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:58:25.051314    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9ca4afb_55ce_40b2_9205_dfa800f39865.slice/crio-407e7ff93c3a12a23f3f03447f11aa2da0c63f1d59f73596ee401a04b46dd936 WatchSource:0}: Error finding container 407e7ff93c3a12a23f3f03447f11aa2da0c63f1d59f73596ee401a04b46dd936: Status 404 returned error can't find the container with id 407e7ff93c3a12a23f3f03447f11aa2da0c63f1d59f73596ee401a04b46dd936
Jul 07 11:58:25.053461 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:25.053416    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 11:58:25.053555 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:25.053541    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 11:58:25.053603 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:25.053587    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 11:58:25.786657 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:25.786621    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch" event={"ID":"c9ca4afb-55ce-40b2-9205-dfa800f39865","Type":"ContainerStarted","Data":"6e3227add20cd51dcde635d0fdd13cbc3a5ea95d24cc155d58b696e25f192781"}
Jul 07 11:58:25.787053 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:25.786681    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch" event={"ID":"c9ca4afb-55ce-40b2-9205-dfa800f39865","Type":"ContainerStarted","Data":"407e7ff93c3a12a23f3f03447f11aa2da0c63f1d59f73596ee401a04b46dd936"}
Jul 07 11:58:25.809635 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:25.809555    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch" podStartSLOduration=1.809539267 podStartE2EDuration="1.809539267s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:58:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:58:25.808352768 +0000 UTC m=+1893.781993218" watchObservedRunningTime="2026-07-07 11:58:25.809539267 +0000 UTC m=+1893.783179677"
Jul 07 11:58:25.906998 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:25.906954    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch"
Jul 07 11:58:26.912840 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:26.912808    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch"
Jul 07 11:58:27.798320 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:27.798287    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch"
Jul 07 11:58:27.799392 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:27.799366    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch"
Jul 07 11:58:40.873127 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:40.873089    2570 generic.go:356] "Generic (PLEG): container finished" podID="c9ca4afb-55ce-40b2-9205-dfa800f39865" containerID="6e3227add20cd51dcde635d0fdd13cbc3a5ea95d24cc155d58b696e25f192781" exitCode=0
Jul 07 11:58:40.873585 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:40.873164    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch" event={"ID":"c9ca4afb-55ce-40b2-9205-dfa800f39865","Type":"ContainerDied","Data":"6e3227add20cd51dcde635d0fdd13cbc3a5ea95d24cc155d58b696e25f192781"}
Jul 07 11:58:40.873630 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:40.873596    2570 scope.go:117] "RemoveContainer" containerID="6e3227add20cd51dcde635d0fdd13cbc3a5ea95d24cc155d58b696e25f192781"
Jul 07 11:58:40.874667 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:40.874638    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 11:58:40.874774 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:40.874722    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 11:58:40.874838 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:40.874771    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 11:58:41.880145 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:41.880109    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch" event={"ID":"c9ca4afb-55ce-40b2-9205-dfa800f39865","Type":"ContainerStarted","Data":"90e8f22d96b476ad0e766669c6cbdf6d3d9c4010f111632cf7ab59cbb6057a91"}
Jul 07 11:58:41.907519 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:41.907491    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch"
Jul 07 11:58:41.908536 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:41.908515    2570 patch_prober.go:28] interesting pod/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Jul 07 11:58:41.908650 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:41.908569    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch" podUID="c9ca4afb-55ce-40b2-9205-dfa800f39865" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Jul 07 11:58:42.799266 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:42.799231    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch"
Jul 07 11:58:42.907706 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:42.907670    2570 patch_prober.go:28] interesting pod/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Jul 07 11:58:42.908091 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:42.907737    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch" podUID="c9ca4afb-55ce-40b2-9205-dfa800f39865" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Jul 07 11:58:43.907525 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:43.907481    2570 patch_prober.go:28] interesting pod/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Jul 07 11:58:43.907760 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:43.907555    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch" podUID="c9ca4afb-55ce-40b2-9205-dfa800f39865" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Jul 07 11:58:44.907399 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:44.907349    2570 patch_prober.go:28] interesting pod/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Jul 07 11:58:44.907590 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:44.907429    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch" podUID="c9ca4afb-55ce-40b2-9205-dfa800f39865" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Jul 07 11:58:45.907292 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:45.907255    2570 patch_prober.go:28] interesting pod/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Jul 07 11:58:45.907804 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:45.907311    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch" podUID="c9ca4afb-55ce-40b2-9205-dfa800f39865" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Jul 07 11:58:45.969040 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:45.969007    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch"]
Jul 07 11:58:45.969257 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:45.969221    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch" podUID="c9ca4afb-55ce-40b2-9205-dfa800f39865" containerName="istio-proxy" containerID="cri-o://90e8f22d96b476ad0e766669c6cbdf6d3d9c4010f111632cf7ab59cbb6057a91" gracePeriod=30
Jul 07 11:58:51.626279 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:51.626251    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch"
Jul 07 11:58:51.717274 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:51.717205    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9ca4afb-55ce-40b2-9205-dfa800f39865-workload-socket\") pod \"c9ca4afb-55ce-40b2-9205-dfa800f39865\" (UID: \"c9ca4afb-55ce-40b2-9205-dfa800f39865\") "
Jul 07 11:58:51.717455 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:51.717306    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9ca4afb-55ce-40b2-9205-dfa800f39865-istio-data\") pod \"c9ca4afb-55ce-40b2-9205-dfa800f39865\" (UID: \"c9ca4afb-55ce-40b2-9205-dfa800f39865\") "
Jul 07 11:58:51.717455 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:51.717362    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9ca4afb-55ce-40b2-9205-dfa800f39865-istio-token\") pod \"c9ca4afb-55ce-40b2-9205-dfa800f39865\" (UID: \"c9ca4afb-55ce-40b2-9205-dfa800f39865\") "
Jul 07 11:58:51.717455 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:51.717386    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9ca4afb-55ce-40b2-9205-dfa800f39865-credential-socket\") pod \"c9ca4afb-55ce-40b2-9205-dfa800f39865\" (UID: \"c9ca4afb-55ce-40b2-9205-dfa800f39865\") "
Jul 07 11:58:51.717455 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:51.717414    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dgffw\" (UniqueName: \"kubernetes.io/projected/c9ca4afb-55ce-40b2-9205-dfa800f39865-kube-api-access-dgffw\") pod \"c9ca4afb-55ce-40b2-9205-dfa800f39865\" (UID: \"c9ca4afb-55ce-40b2-9205-dfa800f39865\") "
Jul 07 11:58:51.717455 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:51.717447    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9ca4afb-55ce-40b2-9205-dfa800f39865-workload-certs\") pod \"c9ca4afb-55ce-40b2-9205-dfa800f39865\" (UID: \"c9ca4afb-55ce-40b2-9205-dfa800f39865\") "
Jul 07 11:58:51.717729 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:51.717449    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9ca4afb-55ce-40b2-9205-dfa800f39865-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c9ca4afb-55ce-40b2-9205-dfa800f39865" (UID: "c9ca4afb-55ce-40b2-9205-dfa800f39865"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:58:51.717729 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:51.717482    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9ca4afb-55ce-40b2-9205-dfa800f39865-istiod-ca-cert\") pod \"c9ca4afb-55ce-40b2-9205-dfa800f39865\" (UID: \"c9ca4afb-55ce-40b2-9205-dfa800f39865\") "
Jul 07 11:58:51.717729 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:51.717550    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9ca4afb-55ce-40b2-9205-dfa800f39865-istio-podinfo\") pod \"c9ca4afb-55ce-40b2-9205-dfa800f39865\" (UID: \"c9ca4afb-55ce-40b2-9205-dfa800f39865\") "
Jul 07 11:58:51.717729 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:51.717574    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9ca4afb-55ce-40b2-9205-dfa800f39865-istio-envoy\") pod \"c9ca4afb-55ce-40b2-9205-dfa800f39865\" (UID: \"c9ca4afb-55ce-40b2-9205-dfa800f39865\") "
Jul 07 11:58:51.717950 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:51.717733    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9ca4afb-55ce-40b2-9205-dfa800f39865-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c9ca4afb-55ce-40b2-9205-dfa800f39865" (UID: "c9ca4afb-55ce-40b2-9205-dfa800f39865"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:58:51.717950 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:51.717767    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9ca4afb-55ce-40b2-9205-dfa800f39865-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c9ca4afb-55ce-40b2-9205-dfa800f39865" (UID: "c9ca4afb-55ce-40b2-9205-dfa800f39865"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:58:51.717950 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:51.717927    2570 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9ca4afb-55ce-40b2-9205-dfa800f39865-credential-socket\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:58:51.717950 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:51.717948    2570 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9ca4afb-55ce-40b2-9205-dfa800f39865-workload-certs\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:58:51.718156 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:51.717963    2570 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9ca4afb-55ce-40b2-9205-dfa800f39865-workload-socket\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:58:51.718156 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:51.717920    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9ca4afb-55ce-40b2-9205-dfa800f39865-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c9ca4afb-55ce-40b2-9205-dfa800f39865" (UID: "c9ca4afb-55ce-40b2-9205-dfa800f39865"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 11:58:51.718271 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:51.718189    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9ca4afb-55ce-40b2-9205-dfa800f39865-istio-data" (OuterVolumeSpecName: "istio-data") pod "c9ca4afb-55ce-40b2-9205-dfa800f39865" (UID: "c9ca4afb-55ce-40b2-9205-dfa800f39865"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:58:51.719684 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:51.719652    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9ca4afb-55ce-40b2-9205-dfa800f39865-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c9ca4afb-55ce-40b2-9205-dfa800f39865" (UID: "c9ca4afb-55ce-40b2-9205-dfa800f39865"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 11:58:51.719684 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:51.719670    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9ca4afb-55ce-40b2-9205-dfa800f39865-kube-api-access-dgffw" (OuterVolumeSpecName: "kube-api-access-dgffw") pod "c9ca4afb-55ce-40b2-9205-dfa800f39865" (UID: "c9ca4afb-55ce-40b2-9205-dfa800f39865"). InnerVolumeSpecName "kube-api-access-dgffw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:58:51.719940 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:51.719921    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9ca4afb-55ce-40b2-9205-dfa800f39865-istio-token" (OuterVolumeSpecName: "istio-token") pod "c9ca4afb-55ce-40b2-9205-dfa800f39865" (UID: "c9ca4afb-55ce-40b2-9205-dfa800f39865"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 11:58:51.720003 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:51.719925    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c9ca4afb-55ce-40b2-9205-dfa800f39865-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c9ca4afb-55ce-40b2-9205-dfa800f39865" (UID: "c9ca4afb-55ce-40b2-9205-dfa800f39865"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 11:58:51.819274 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:51.819237    2570 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9ca4afb-55ce-40b2-9205-dfa800f39865-istio-data\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:58:51.819274 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:51.819271    2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9ca4afb-55ce-40b2-9205-dfa800f39865-istio-token\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:58:51.819547 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:51.819286    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dgffw\" (UniqueName: \"kubernetes.io/projected/c9ca4afb-55ce-40b2-9205-dfa800f39865-kube-api-access-dgffw\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:58:51.819547 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:51.819298    2570 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9ca4afb-55ce-40b2-9205-dfa800f39865-istiod-ca-cert\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:58:51.819547 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:51.819307    2570 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9ca4afb-55ce-40b2-9205-dfa800f39865-istio-podinfo\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:58:51.819547 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:51.819315    2570 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9ca4afb-55ce-40b2-9205-dfa800f39865-istio-envoy\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 11:58:51.935682 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:51.935646    2570 generic.go:356] "Generic (PLEG): container finished" podID="c9ca4afb-55ce-40b2-9205-dfa800f39865" containerID="90e8f22d96b476ad0e766669c6cbdf6d3d9c4010f111632cf7ab59cbb6057a91" exitCode=0
Jul 07 11:58:51.935937 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:51.935783    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch"
Jul 07 11:58:51.938628 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:51.938598    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch" event={"ID":"c9ca4afb-55ce-40b2-9205-dfa800f39865","Type":"ContainerDied","Data":"90e8f22d96b476ad0e766669c6cbdf6d3d9c4010f111632cf7ab59cbb6057a91"}
Jul 07 11:58:51.938794 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:51.938647    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch" event={"ID":"c9ca4afb-55ce-40b2-9205-dfa800f39865","Type":"ContainerDied","Data":"407e7ff93c3a12a23f3f03447f11aa2da0c63f1d59f73596ee401a04b46dd936"}
Jul 07 11:58:51.938794 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:51.938671    2570 scope.go:117] "RemoveContainer" containerID="90e8f22d96b476ad0e766669c6cbdf6d3d9c4010f111632cf7ab59cbb6057a91"
Jul 07 11:58:51.948614 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:51.948594    2570 scope.go:117] "RemoveContainer" containerID="6e3227add20cd51dcde635d0fdd13cbc3a5ea95d24cc155d58b696e25f192781"
Jul 07 11:58:51.957387 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:51.957361    2570 scope.go:117] "RemoveContainer" containerID="90e8f22d96b476ad0e766669c6cbdf6d3d9c4010f111632cf7ab59cbb6057a91"
Jul 07 11:58:51.957815 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:58:51.957786    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90e8f22d96b476ad0e766669c6cbdf6d3d9c4010f111632cf7ab59cbb6057a91\": container with ID starting with 90e8f22d96b476ad0e766669c6cbdf6d3d9c4010f111632cf7ab59cbb6057a91 not found: ID does not exist" containerID="90e8f22d96b476ad0e766669c6cbdf6d3d9c4010f111632cf7ab59cbb6057a91"
Jul 07 11:58:51.957927 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:51.957825    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90e8f22d96b476ad0e766669c6cbdf6d3d9c4010f111632cf7ab59cbb6057a91"} err="failed to get container status \"90e8f22d96b476ad0e766669c6cbdf6d3d9c4010f111632cf7ab59cbb6057a91\": rpc error: code = NotFound desc = could not find container \"90e8f22d96b476ad0e766669c6cbdf6d3d9c4010f111632cf7ab59cbb6057a91\": container with ID starting with 90e8f22d96b476ad0e766669c6cbdf6d3d9c4010f111632cf7ab59cbb6057a91 not found: ID does not exist"
Jul 07 11:58:51.957927 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:51.957850    2570 scope.go:117] "RemoveContainer" containerID="6e3227add20cd51dcde635d0fdd13cbc3a5ea95d24cc155d58b696e25f192781"
Jul 07 11:58:51.958116 ip-10-0-135-78 kubenswrapper[2570]: E0707 11:58:51.958096    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e3227add20cd51dcde635d0fdd13cbc3a5ea95d24cc155d58b696e25f192781\": container with ID starting with 6e3227add20cd51dcde635d0fdd13cbc3a5ea95d24cc155d58b696e25f192781 not found: ID does not exist" containerID="6e3227add20cd51dcde635d0fdd13cbc3a5ea95d24cc155d58b696e25f192781"
Jul 07 11:58:51.958238 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:51.958124    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e3227add20cd51dcde635d0fdd13cbc3a5ea95d24cc155d58b696e25f192781"} err="failed to get container status \"6e3227add20cd51dcde635d0fdd13cbc3a5ea95d24cc155d58b696e25f192781\": rpc error: code = NotFound desc = could not find container \"6e3227add20cd51dcde635d0fdd13cbc3a5ea95d24cc155d58b696e25f192781\": container with ID starting with 6e3227add20cd51dcde635d0fdd13cbc3a5ea95d24cc155d58b696e25f192781 not found: ID does not exist"
Jul 07 11:58:51.960533 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:51.960509    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch"]
Jul 07 11:58:51.966242 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:51.966200    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-5e15070c-openshift-default-f49f7ff87-zlcch"]
Jul 07 11:58:53.938827 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:58:53.938794    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9ca4afb-55ce-40b2-9205-dfa800f39865" path="/var/lib/kubelet/pods/c9ca4afb-55ce-40b2-9205-dfa800f39865/volumes"
Jul 07 11:59:05.262438 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:05.262402    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9"]
Jul 07 11:59:05.263017 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:05.262943    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9ca4afb-55ce-40b2-9205-dfa800f39865" containerName="istio-proxy"
Jul 07 11:59:05.263017 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:05.262965    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9ca4afb-55ce-40b2-9205-dfa800f39865" containerName="istio-proxy"
Jul 07 11:59:05.263017 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:05.262977    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9ca4afb-55ce-40b2-9205-dfa800f39865" containerName="istio-proxy"
Jul 07 11:59:05.263017 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:05.262987    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9ca4afb-55ce-40b2-9205-dfa800f39865" containerName="istio-proxy"
Jul 07 11:59:05.263315 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:05.263080    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9ca4afb-55ce-40b2-9205-dfa800f39865" containerName="istio-proxy"
Jul 07 11:59:05.263315 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:05.263096    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9ca4afb-55ce-40b2-9205-dfa800f39865" containerName="istio-proxy"
Jul 07 11:59:05.266282 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:05.266250    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9"
Jul 07 11:59:05.269194 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:05.269174    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-b80bcdc6-openshift-default-dockercfg-gkk6x\""
Jul 07 11:59:05.281571 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:05.281543    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9"]
Jul 07 11:59:05.345966 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:05.345899    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/53a5831a-7072-4369-9820-12aa1886dd84-istio-token\") pod \"e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9\" (UID: \"53a5831a-7072-4369-9820-12aa1886dd84\") " pod="openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9"
Jul 07 11:59:05.345966 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:05.345965    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/53a5831a-7072-4369-9820-12aa1886dd84-istio-podinfo\") pod \"e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9\" (UID: \"53a5831a-7072-4369-9820-12aa1886dd84\") " pod="openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9"
Jul 07 11:59:05.346312 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:05.345998    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/53a5831a-7072-4369-9820-12aa1886dd84-credential-socket\") pod \"e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9\" (UID: \"53a5831a-7072-4369-9820-12aa1886dd84\") " pod="openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9"
Jul 07 11:59:05.346312 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:05.346068    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/53a5831a-7072-4369-9820-12aa1886dd84-istio-data\") pod \"e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9\" (UID: \"53a5831a-7072-4369-9820-12aa1886dd84\") " pod="openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9"
Jul 07 11:59:05.346312 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:05.346174    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhdqh\" (UniqueName: \"kubernetes.io/projected/53a5831a-7072-4369-9820-12aa1886dd84-kube-api-access-dhdqh\") pod \"e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9\" (UID: \"53a5831a-7072-4369-9820-12aa1886dd84\") " pod="openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9"
Jul 07 11:59:05.346312 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:05.346259    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/53a5831a-7072-4369-9820-12aa1886dd84-workload-certs\") pod \"e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9\" (UID: \"53a5831a-7072-4369-9820-12aa1886dd84\") " pod="openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9"
Jul 07 11:59:05.346312 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:05.346300    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/53a5831a-7072-4369-9820-12aa1886dd84-istio-envoy\") pod \"e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9\" (UID: \"53a5831a-7072-4369-9820-12aa1886dd84\") " pod="openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9"
Jul 07 11:59:05.346533 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:05.346345    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/53a5831a-7072-4369-9820-12aa1886dd84-workload-socket\") pod \"e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9\" (UID: \"53a5831a-7072-4369-9820-12aa1886dd84\") " pod="openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9"
Jul 07 11:59:05.346533 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:05.346370    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/53a5831a-7072-4369-9820-12aa1886dd84-istiod-ca-cert\") pod \"e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9\" (UID: \"53a5831a-7072-4369-9820-12aa1886dd84\") " pod="openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9"
Jul 07 11:59:05.447285 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:05.447238    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/53a5831a-7072-4369-9820-12aa1886dd84-workload-certs\") pod \"e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9\" (UID: \"53a5831a-7072-4369-9820-12aa1886dd84\") " pod="openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9"
Jul 07 11:59:05.447514 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:05.447308    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/53a5831a-7072-4369-9820-12aa1886dd84-istio-envoy\") pod \"e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9\" (UID: \"53a5831a-7072-4369-9820-12aa1886dd84\") " pod="openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9"
Jul 07 11:59:05.447514 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:05.447351    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/53a5831a-7072-4369-9820-12aa1886dd84-workload-socket\") pod \"e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9\" (UID: \"53a5831a-7072-4369-9820-12aa1886dd84\") " pod="openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9"
Jul 07 11:59:05.447514 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:05.447374    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/53a5831a-7072-4369-9820-12aa1886dd84-istiod-ca-cert\") pod \"e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9\" (UID: \"53a5831a-7072-4369-9820-12aa1886dd84\") " pod="openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9"
Jul 07 11:59:05.447514 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:05.447433    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/53a5831a-7072-4369-9820-12aa1886dd84-istio-token\") pod \"e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9\" (UID: \"53a5831a-7072-4369-9820-12aa1886dd84\") " pod="openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9"
Jul 07 11:59:05.447514 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:05.447467    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/53a5831a-7072-4369-9820-12aa1886dd84-istio-podinfo\") pod \"e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9\" (UID: \"53a5831a-7072-4369-9820-12aa1886dd84\") " pod="openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9"
Jul 07 11:59:05.447514 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:05.447513    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/53a5831a-7072-4369-9820-12aa1886dd84-credential-socket\") pod \"e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9\" (UID: \"53a5831a-7072-4369-9820-12aa1886dd84\") " pod="openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9"
Jul 07 11:59:05.447851 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:05.447542    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/53a5831a-7072-4369-9820-12aa1886dd84-istio-data\") pod \"e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9\" (UID: \"53a5831a-7072-4369-9820-12aa1886dd84\") " pod="openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9"
Jul 07 11:59:05.448120 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:05.447707    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/53a5831a-7072-4369-9820-12aa1886dd84-workload-certs\") pod \"e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9\" (UID: \"53a5831a-7072-4369-9820-12aa1886dd84\") " pod="openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9"
Jul 07 11:59:05.448120 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:05.447728    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/53a5831a-7072-4369-9820-12aa1886dd84-workload-socket\") pod \"e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9\" (UID: \"53a5831a-7072-4369-9820-12aa1886dd84\") " pod="openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9"
Jul 07 11:59:05.448120 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:05.447981    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhdqh\" (UniqueName: \"kubernetes.io/projected/53a5831a-7072-4369-9820-12aa1886dd84-kube-api-access-dhdqh\") pod \"e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9\" (UID: \"53a5831a-7072-4369-9820-12aa1886dd84\") " pod="openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9"
Jul 07 11:59:05.448120 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:05.448063    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/53a5831a-7072-4369-9820-12aa1886dd84-credential-socket\") pod \"e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9\" (UID: \"53a5831a-7072-4369-9820-12aa1886dd84\") " pod="openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9"
Jul 07 11:59:05.448120 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:05.448109    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/53a5831a-7072-4369-9820-12aa1886dd84-istio-data\") pod \"e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9\" (UID: \"53a5831a-7072-4369-9820-12aa1886dd84\") " pod="openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9"
Jul 07 11:59:05.448423 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:05.448134    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/53a5831a-7072-4369-9820-12aa1886dd84-istiod-ca-cert\") pod \"e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9\" (UID: \"53a5831a-7072-4369-9820-12aa1886dd84\") " pod="openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9"
Jul 07 11:59:05.449867 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:05.449845    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/53a5831a-7072-4369-9820-12aa1886dd84-istio-envoy\") pod \"e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9\" (UID: \"53a5831a-7072-4369-9820-12aa1886dd84\") " pod="openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9"
Jul 07 11:59:05.450276 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:05.450255    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/53a5831a-7072-4369-9820-12aa1886dd84-istio-podinfo\") pod \"e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9\" (UID: \"53a5831a-7072-4369-9820-12aa1886dd84\") " pod="openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9"
Jul 07 11:59:05.456034 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:05.456003    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/53a5831a-7072-4369-9820-12aa1886dd84-istio-token\") pod \"e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9\" (UID: \"53a5831a-7072-4369-9820-12aa1886dd84\") " pod="openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9"
Jul 07 11:59:05.456271 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:05.456246    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhdqh\" (UniqueName: \"kubernetes.io/projected/53a5831a-7072-4369-9820-12aa1886dd84-kube-api-access-dhdqh\") pod \"e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9\" (UID: \"53a5831a-7072-4369-9820-12aa1886dd84\") " pod="openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9"
Jul 07 11:59:05.577013 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:05.576922    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9"
Jul 07 11:59:05.718426 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:05.718395    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9"]
Jul 07 11:59:05.718939 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:59:05.718913    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53a5831a_7072_4369_9820_12aa1886dd84.slice/crio-3e4826dbd3cc87edd310a2bb0aaa7a54c46a48e7950de67ca1435ef04697f8a4 WatchSource:0}: Error finding container 3e4826dbd3cc87edd310a2bb0aaa7a54c46a48e7950de67ca1435ef04697f8a4: Status 404 returned error can't find the container with id 3e4826dbd3cc87edd310a2bb0aaa7a54c46a48e7950de67ca1435ef04697f8a4
Jul 07 11:59:05.721278 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:05.721249    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 11:59:05.721357 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:05.721310    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 11:59:05.721357 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:05.721339    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 11:59:06.009435 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:06.009398    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9" event={"ID":"53a5831a-7072-4369-9820-12aa1886dd84","Type":"ContainerStarted","Data":"133f0a0f2b936a9f6adba2f179d60a2a99b9c600c4d4b26fbbbee60a84c3dddd"}
Jul 07 11:59:06.009654 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:06.009463    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9" event={"ID":"53a5831a-7072-4369-9820-12aa1886dd84","Type":"ContainerStarted","Data":"3e4826dbd3cc87edd310a2bb0aaa7a54c46a48e7950de67ca1435ef04697f8a4"}
Jul 07 11:59:06.037901 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:06.037839    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9" podStartSLOduration=1.037821084 podStartE2EDuration="1.037821084s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:59:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:59:06.037119704 +0000 UTC m=+1934.010760138" watchObservedRunningTime="2026-07-07 11:59:06.037821084 +0000 UTC m=+1934.011461494"
Jul 07 11:59:06.577647 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:06.577604    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9"
Jul 07 11:59:06.582438 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:06.582412    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9"
Jul 07 11:59:07.015762 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:07.015729    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9"
Jul 07 11:59:07.017060 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:07.017035    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9"
Jul 07 11:59:22.926357 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:22.926302    2570 patch_prober.go:28] interesting pod/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": read tcp 10.132.0.2:56370->10.132.0.44:15021: read: connection reset by peer" start-of-body=
Jul 07 11:59:22.926758 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:22.926381    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9" podUID="53a5831a-7072-4369-9820-12aa1886dd84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": read tcp 10.132.0.2:56370->10.132.0.44:15021: read: connection reset by peer"
Jul 07 11:59:23.105116 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:23.105030    2570 generic.go:356] "Generic (PLEG): container finished" podID="53a5831a-7072-4369-9820-12aa1886dd84" containerID="133f0a0f2b936a9f6adba2f179d60a2a99b9c600c4d4b26fbbbee60a84c3dddd" exitCode=0
Jul 07 11:59:23.105116 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:23.105106    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9" event={"ID":"53a5831a-7072-4369-9820-12aa1886dd84","Type":"ContainerDied","Data":"133f0a0f2b936a9f6adba2f179d60a2a99b9c600c4d4b26fbbbee60a84c3dddd"}
Jul 07 11:59:23.105544 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:23.105529    2570 scope.go:117] "RemoveContainer" containerID="133f0a0f2b936a9f6adba2f179d60a2a99b9c600c4d4b26fbbbee60a84c3dddd"
Jul 07 11:59:23.106553 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:23.106523    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 11:59:23.106609 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:23.106595    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 11:59:23.106645 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:23.106635    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 11:59:24.112206 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:24.112170    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9" event={"ID":"53a5831a-7072-4369-9820-12aa1886dd84","Type":"ContainerStarted","Data":"f0b0a4a477518187319a77324e173d051a53f39ab934092914ef3e4bb730b978"}
Jul 07 11:59:24.578117 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:24.578066    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9"
Jul 07 11:59:24.578429 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:24.578404    2570 patch_prober.go:28] interesting pod/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Jul 07 11:59:24.578483 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:24.578459    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9" podUID="53a5831a-7072-4369-9820-12aa1886dd84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Jul 07 11:59:25.578401 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:25.578355    2570 patch_prober.go:28] interesting pod/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Jul 07 11:59:25.578918 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:25.578464    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9" podUID="53a5831a-7072-4369-9820-12aa1886dd84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Jul 07 11:59:26.578396 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:26.578355    2570 patch_prober.go:28] interesting pod/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Jul 07 11:59:26.578593 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:26.578416    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9" podUID="53a5831a-7072-4369-9820-12aa1886dd84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Jul 07 11:59:27.320741 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:27.320692    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-b80bcdc6/e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz"]
Jul 07 11:59:27.331046 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:27.331012    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-b80bcdc6/e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz"
Jul 07 11:59:27.343232 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:27.339143    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-b80bcdc6/e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz"]
Jul 07 11:59:27.348233 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:27.345973    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-b80bcdc6\"/\"e2e-lifecycle-model-b80bcdc6-kserve-self-signed-certs\""
Jul 07 11:59:27.348233 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:27.346290    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-b80bcdc6\"/\"openshift-service-ca.crt\""
Jul 07 11:59:27.348233 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:27.346506    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-b80bcdc6\"/\"default-dockercfg-c4tvk\""
Jul 07 11:59:27.348233 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:27.346728    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-b80bcdc6\"/\"kube-root-ca.crt\""
Jul 07 11:59:27.364637 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:27.364603    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0620ae1-01ac-437e-bcb2-9f85957e0abc-tls-certs\") pod \"e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz\" (UID: \"e0620ae1-01ac-437e-bcb2-9f85957e0abc\") " pod="ai-tenant-e2e-mt-b80bcdc6/e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz"
Jul 07 11:59:27.364637 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:27.364638    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0620ae1-01ac-437e-bcb2-9f85957e0abc-tmp-dir\") pod \"e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz\" (UID: \"e0620ae1-01ac-437e-bcb2-9f85957e0abc\") " pod="ai-tenant-e2e-mt-b80bcdc6/e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz"
Jul 07 11:59:27.364897 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:27.364683    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blxwz\" (UniqueName: \"kubernetes.io/projected/e0620ae1-01ac-437e-bcb2-9f85957e0abc-kube-api-access-blxwz\") pod \"e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz\" (UID: \"e0620ae1-01ac-437e-bcb2-9f85957e0abc\") " pod="ai-tenant-e2e-mt-b80bcdc6/e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz"
Jul 07 11:59:27.364897 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:27.364712    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0620ae1-01ac-437e-bcb2-9f85957e0abc-model-cache\") pod \"e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz\" (UID: \"e0620ae1-01ac-437e-bcb2-9f85957e0abc\") " pod="ai-tenant-e2e-mt-b80bcdc6/e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz"
Jul 07 11:59:27.364897 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:27.364803    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0620ae1-01ac-437e-bcb2-9f85957e0abc-home\") pod \"e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz\" (UID: \"e0620ae1-01ac-437e-bcb2-9f85957e0abc\") " pod="ai-tenant-e2e-mt-b80bcdc6/e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz"
Jul 07 11:59:27.364897 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:27.364853    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0620ae1-01ac-437e-bcb2-9f85957e0abc-dshm\") pod \"e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz\" (UID: \"e0620ae1-01ac-437e-bcb2-9f85957e0abc\") " pod="ai-tenant-e2e-mt-b80bcdc6/e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz"
Jul 07 11:59:27.465900 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:27.465863    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0620ae1-01ac-437e-bcb2-9f85957e0abc-tls-certs\") pod \"e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz\" (UID: \"e0620ae1-01ac-437e-bcb2-9f85957e0abc\") " pod="ai-tenant-e2e-mt-b80bcdc6/e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz"
Jul 07 11:59:27.465900 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:27.465904    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0620ae1-01ac-437e-bcb2-9f85957e0abc-tmp-dir\") pod \"e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz\" (UID: \"e0620ae1-01ac-437e-bcb2-9f85957e0abc\") " pod="ai-tenant-e2e-mt-b80bcdc6/e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz"
Jul 07 11:59:27.466172 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:27.465933    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blxwz\" (UniqueName: \"kubernetes.io/projected/e0620ae1-01ac-437e-bcb2-9f85957e0abc-kube-api-access-blxwz\") pod \"e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz\" (UID: \"e0620ae1-01ac-437e-bcb2-9f85957e0abc\") " pod="ai-tenant-e2e-mt-b80bcdc6/e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz"
Jul 07 11:59:27.466172 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:27.465962    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0620ae1-01ac-437e-bcb2-9f85957e0abc-model-cache\") pod \"e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz\" (UID: \"e0620ae1-01ac-437e-bcb2-9f85957e0abc\") " pod="ai-tenant-e2e-mt-b80bcdc6/e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz"
Jul 07 11:59:27.466172 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:27.466042    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0620ae1-01ac-437e-bcb2-9f85957e0abc-home\") pod \"e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz\" (UID: \"e0620ae1-01ac-437e-bcb2-9f85957e0abc\") " pod="ai-tenant-e2e-mt-b80bcdc6/e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz"
Jul 07 11:59:27.466172 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:27.466106    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0620ae1-01ac-437e-bcb2-9f85957e0abc-dshm\") pod \"e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz\" (UID: \"e0620ae1-01ac-437e-bcb2-9f85957e0abc\") " pod="ai-tenant-e2e-mt-b80bcdc6/e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz"
Jul 07 11:59:27.466423 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:27.466397    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0620ae1-01ac-437e-bcb2-9f85957e0abc-tmp-dir\") pod \"e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz\" (UID: \"e0620ae1-01ac-437e-bcb2-9f85957e0abc\") " pod="ai-tenant-e2e-mt-b80bcdc6/e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz"
Jul 07 11:59:27.466423 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:27.466411    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0620ae1-01ac-437e-bcb2-9f85957e0abc-model-cache\") pod \"e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz\" (UID: \"e0620ae1-01ac-437e-bcb2-9f85957e0abc\") " pod="ai-tenant-e2e-mt-b80bcdc6/e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz"
Jul 07 11:59:27.466616 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:27.466591    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0620ae1-01ac-437e-bcb2-9f85957e0abc-home\") pod \"e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz\" (UID: \"e0620ae1-01ac-437e-bcb2-9f85957e0abc\") " pod="ai-tenant-e2e-mt-b80bcdc6/e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz"
Jul 07 11:59:27.468376 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:27.468353    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0620ae1-01ac-437e-bcb2-9f85957e0abc-dshm\") pod \"e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz\" (UID: \"e0620ae1-01ac-437e-bcb2-9f85957e0abc\") " pod="ai-tenant-e2e-mt-b80bcdc6/e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz"
Jul 07 11:59:27.468621 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:27.468599    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0620ae1-01ac-437e-bcb2-9f85957e0abc-tls-certs\") pod \"e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz\" (UID: \"e0620ae1-01ac-437e-bcb2-9f85957e0abc\") " pod="ai-tenant-e2e-mt-b80bcdc6/e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz"
Jul 07 11:59:27.473933 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:27.473902    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-blxwz\" (UniqueName: \"kubernetes.io/projected/e0620ae1-01ac-437e-bcb2-9f85957e0abc-kube-api-access-blxwz\") pod \"e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz\" (UID: \"e0620ae1-01ac-437e-bcb2-9f85957e0abc\") " pod="ai-tenant-e2e-mt-b80bcdc6/e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz"
Jul 07 11:59:27.578499 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:27.578399    2570 patch_prober.go:28] interesting pod/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Jul 07 11:59:27.578499 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:27.578459    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9" podUID="53a5831a-7072-4369-9820-12aa1886dd84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Jul 07 11:59:27.656418 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:27.656370    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-b80bcdc6/e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz"
Jul 07 11:59:27.847325 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:27.847286    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-b80bcdc6/e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz"]
Jul 07 11:59:27.849687 ip-10-0-135-78 kubenswrapper[2570]: W0707 11:59:27.849647    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode0620ae1_01ac_437e_bcb2_9f85957e0abc.slice/crio-70fcab6b034f048d6d89f24779e6c61bac412d175075d45927f61ea446d08a7f WatchSource:0}: Error finding container 70fcab6b034f048d6d89f24779e6c61bac412d175075d45927f61ea446d08a7f: Status 404 returned error can't find the container with id 70fcab6b034f048d6d89f24779e6c61bac412d175075d45927f61ea446d08a7f
Jul 07 11:59:28.146510 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:28.146414    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-b80bcdc6/e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz" event={"ID":"e0620ae1-01ac-437e-bcb2-9f85957e0abc","Type":"ContainerStarted","Data":"798295031d8682f365f17e9a17cb1fcbd2a30a887903e2d1f83cce6d038a482b"}
Jul 07 11:59:28.146510 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:28.146466    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-b80bcdc6/e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz" event={"ID":"e0620ae1-01ac-437e-bcb2-9f85957e0abc","Type":"ContainerStarted","Data":"70fcab6b034f048d6d89f24779e6c61bac412d175075d45927f61ea446d08a7f"}
Jul 07 11:59:28.166466 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:28.166408    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-b80bcdc6/e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz" podStartSLOduration=1.166385685 podStartE2EDuration="1.166385685s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:59:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:59:28.163527793 +0000 UTC m=+1956.137168203" watchObservedRunningTime="2026-07-07 11:59:28.166385685 +0000 UTC m=+1956.140026095"
Jul 07 11:59:28.580796 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:28.580764    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9"
Jul 07 11:59:28.581259 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:28.581017    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9"
Jul 07 11:59:28.581710 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:28.581692    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9"
Jul 07 11:59:37.657290 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:37.657247    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-b80bcdc6/e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz"
Jul 07 11:59:37.657714 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:37.657308    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-b80bcdc6/e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz"
Jul 07 11:59:37.670086 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:37.670059    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-b80bcdc6/e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz"
Jul 07 11:59:38.217220 ip-10-0-135-78 kubenswrapper[2570]: I0707 11:59:38.217169    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-b80bcdc6/e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz"
Jul 07 12:00:01.096266 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:01.096233    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-b80bcdc6/e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz"]
Jul 07 12:00:16.117878 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:16.117807    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-b80bcdc6/e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz" podUID="e0620ae1-01ac-437e-bcb2-9f85957e0abc" containerName="main" containerID="cri-o://798295031d8682f365f17e9a17cb1fcbd2a30a887903e2d1f83cce6d038a482b" gracePeriod=45
Jul 07 12:00:16.376466 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:16.376400    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-b80bcdc6/e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz"
Jul 07 12:00:16.407271 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:16.407194    2570 generic.go:356] "Generic (PLEG): container finished" podID="e0620ae1-01ac-437e-bcb2-9f85957e0abc" containerID="798295031d8682f365f17e9a17cb1fcbd2a30a887903e2d1f83cce6d038a482b" exitCode=0
Jul 07 12:00:16.407271 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:16.407250    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-b80bcdc6/e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz" event={"ID":"e0620ae1-01ac-437e-bcb2-9f85957e0abc","Type":"ContainerDied","Data":"798295031d8682f365f17e9a17cb1fcbd2a30a887903e2d1f83cce6d038a482b"}
Jul 07 12:00:16.407522 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:16.407308    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-b80bcdc6/e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz" event={"ID":"e0620ae1-01ac-437e-bcb2-9f85957e0abc","Type":"ContainerDied","Data":"70fcab6b034f048d6d89f24779e6c61bac412d175075d45927f61ea446d08a7f"}
Jul 07 12:00:16.407522 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:16.407329    2570 scope.go:117] "RemoveContainer" containerID="798295031d8682f365f17e9a17cb1fcbd2a30a887903e2d1f83cce6d038a482b"
Jul 07 12:00:16.407522 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:16.407326    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-b80bcdc6/e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz"
Jul 07 12:00:16.420089 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:16.420065    2570 scope.go:117] "RemoveContainer" containerID="798295031d8682f365f17e9a17cb1fcbd2a30a887903e2d1f83cce6d038a482b"
Jul 07 12:00:16.420469 ip-10-0-135-78 kubenswrapper[2570]: E0707 12:00:16.420437    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"798295031d8682f365f17e9a17cb1fcbd2a30a887903e2d1f83cce6d038a482b\": container with ID starting with 798295031d8682f365f17e9a17cb1fcbd2a30a887903e2d1f83cce6d038a482b not found: ID does not exist" containerID="798295031d8682f365f17e9a17cb1fcbd2a30a887903e2d1f83cce6d038a482b"
Jul 07 12:00:16.420587 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:16.420479    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"798295031d8682f365f17e9a17cb1fcbd2a30a887903e2d1f83cce6d038a482b"} err="failed to get container status \"798295031d8682f365f17e9a17cb1fcbd2a30a887903e2d1f83cce6d038a482b\": rpc error: code = NotFound desc = could not find container \"798295031d8682f365f17e9a17cb1fcbd2a30a887903e2d1f83cce6d038a482b\": container with ID starting with 798295031d8682f365f17e9a17cb1fcbd2a30a887903e2d1f83cce6d038a482b not found: ID does not exist"
Jul 07 12:00:16.425774 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:16.425749    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0620ae1-01ac-437e-bcb2-9f85957e0abc-dshm\") pod \"e0620ae1-01ac-437e-bcb2-9f85957e0abc\" (UID: \"e0620ae1-01ac-437e-bcb2-9f85957e0abc\") "
Jul 07 12:00:16.425905 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:16.425785    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0620ae1-01ac-437e-bcb2-9f85957e0abc-model-cache\") pod \"e0620ae1-01ac-437e-bcb2-9f85957e0abc\" (UID: \"e0620ae1-01ac-437e-bcb2-9f85957e0abc\") "
Jul 07 12:00:16.425905 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:16.425849    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0620ae1-01ac-437e-bcb2-9f85957e0abc-tls-certs\") pod \"e0620ae1-01ac-437e-bcb2-9f85957e0abc\" (UID: \"e0620ae1-01ac-437e-bcb2-9f85957e0abc\") "
Jul 07 12:00:16.425905 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:16.425873    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-blxwz\" (UniqueName: \"kubernetes.io/projected/e0620ae1-01ac-437e-bcb2-9f85957e0abc-kube-api-access-blxwz\") pod \"e0620ae1-01ac-437e-bcb2-9f85957e0abc\" (UID: \"e0620ae1-01ac-437e-bcb2-9f85957e0abc\") "
Jul 07 12:00:16.425905 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:16.425898    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0620ae1-01ac-437e-bcb2-9f85957e0abc-home\") pod \"e0620ae1-01ac-437e-bcb2-9f85957e0abc\" (UID: \"e0620ae1-01ac-437e-bcb2-9f85957e0abc\") "
Jul 07 12:00:16.426124 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:16.426031    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0620ae1-01ac-437e-bcb2-9f85957e0abc-tmp-dir\") pod \"e0620ae1-01ac-437e-bcb2-9f85957e0abc\" (UID: \"e0620ae1-01ac-437e-bcb2-9f85957e0abc\") "
Jul 07 12:00:16.426124 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:16.426068    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0620ae1-01ac-437e-bcb2-9f85957e0abc-model-cache" (OuterVolumeSpecName: "model-cache") pod "e0620ae1-01ac-437e-bcb2-9f85957e0abc" (UID: "e0620ae1-01ac-437e-bcb2-9f85957e0abc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:00:16.426331 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:16.426277    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0620ae1-01ac-437e-bcb2-9f85957e0abc-home" (OuterVolumeSpecName: "home") pod "e0620ae1-01ac-437e-bcb2-9f85957e0abc" (UID: "e0620ae1-01ac-437e-bcb2-9f85957e0abc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:00:16.426411 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:16.426335    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0620ae1-01ac-437e-bcb2-9f85957e0abc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e0620ae1-01ac-437e-bcb2-9f85957e0abc" (UID: "e0620ae1-01ac-437e-bcb2-9f85957e0abc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:00:16.426411 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:16.426378    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0620ae1-01ac-437e-bcb2-9f85957e0abc-model-cache\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:00:16.428104 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:16.428082    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0620ae1-01ac-437e-bcb2-9f85957e0abc-dshm" (OuterVolumeSpecName: "dshm") pod "e0620ae1-01ac-437e-bcb2-9f85957e0abc" (UID: "e0620ae1-01ac-437e-bcb2-9f85957e0abc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:00:16.428461 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:16.428439    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0620ae1-01ac-437e-bcb2-9f85957e0abc-kube-api-access-blxwz" (OuterVolumeSpecName: "kube-api-access-blxwz") pod "e0620ae1-01ac-437e-bcb2-9f85957e0abc" (UID: "e0620ae1-01ac-437e-bcb2-9f85957e0abc"). InnerVolumeSpecName "kube-api-access-blxwz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 12:00:16.428461 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:16.428446    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0620ae1-01ac-437e-bcb2-9f85957e0abc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e0620ae1-01ac-437e-bcb2-9f85957e0abc" (UID: "e0620ae1-01ac-437e-bcb2-9f85957e0abc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 12:00:16.527543 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:16.527506    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0620ae1-01ac-437e-bcb2-9f85957e0abc-tls-certs\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:00:16.527543 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:16.527540    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-blxwz\" (UniqueName: \"kubernetes.io/projected/e0620ae1-01ac-437e-bcb2-9f85957e0abc-kube-api-access-blxwz\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:00:16.527543 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:16.527550    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0620ae1-01ac-437e-bcb2-9f85957e0abc-home\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:00:16.527801 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:16.527559    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0620ae1-01ac-437e-bcb2-9f85957e0abc-tmp-dir\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:00:16.527801 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:16.527567    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0620ae1-01ac-437e-bcb2-9f85957e0abc-dshm\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:00:16.731330 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:16.731293    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-b80bcdc6/e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz"]
Jul 07 12:00:16.733527 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:16.733506    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-b80bcdc6/e2e-lifecycle-model-b80bcdc6-kserve-668ff74d8c-2xjgz"]
Jul 07 12:00:17.937987 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:17.937954    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0620ae1-01ac-437e-bcb2-9f85957e0abc" path="/var/lib/kubelet/pods/e0620ae1-01ac-437e-bcb2-9f85957e0abc/volumes"
Jul 07 12:00:44.287724 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:44.287682    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9"]
Jul 07 12:00:44.288322 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:44.287940    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9" podUID="53a5831a-7072-4369-9820-12aa1886dd84" containerName="istio-proxy" containerID="cri-o://f0b0a4a477518187319a77324e173d051a53f39ab934092914ef3e4bb730b978" gracePeriod=30
Jul 07 12:00:49.552031 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:49.551967    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9"
Jul 07 12:00:49.587261 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:49.587203    2570 generic.go:356] "Generic (PLEG): container finished" podID="53a5831a-7072-4369-9820-12aa1886dd84" containerID="f0b0a4a477518187319a77324e173d051a53f39ab934092914ef3e4bb730b978" exitCode=0
Jul 07 12:00:49.587491 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:49.587268    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9"
Jul 07 12:00:49.587491 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:49.587285    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9" event={"ID":"53a5831a-7072-4369-9820-12aa1886dd84","Type":"ContainerDied","Data":"f0b0a4a477518187319a77324e173d051a53f39ab934092914ef3e4bb730b978"}
Jul 07 12:00:49.587491 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:49.587353    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9" event={"ID":"53a5831a-7072-4369-9820-12aa1886dd84","Type":"ContainerDied","Data":"3e4826dbd3cc87edd310a2bb0aaa7a54c46a48e7950de67ca1435ef04697f8a4"}
Jul 07 12:00:49.587491 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:49.587378    2570 scope.go:117] "RemoveContainer" containerID="f0b0a4a477518187319a77324e173d051a53f39ab934092914ef3e4bb730b978"
Jul 07 12:00:49.596633 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:49.596613    2570 scope.go:117] "RemoveContainer" containerID="133f0a0f2b936a9f6adba2f179d60a2a99b9c600c4d4b26fbbbee60a84c3dddd"
Jul 07 12:00:49.606895 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:49.606875    2570 scope.go:117] "RemoveContainer" containerID="f0b0a4a477518187319a77324e173d051a53f39ab934092914ef3e4bb730b978"
Jul 07 12:00:49.607158 ip-10-0-135-78 kubenswrapper[2570]: E0707 12:00:49.607139    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0b0a4a477518187319a77324e173d051a53f39ab934092914ef3e4bb730b978\": container with ID starting with f0b0a4a477518187319a77324e173d051a53f39ab934092914ef3e4bb730b978 not found: ID does not exist" containerID="f0b0a4a477518187319a77324e173d051a53f39ab934092914ef3e4bb730b978"
Jul 07 12:00:49.607224 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:49.607172    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0b0a4a477518187319a77324e173d051a53f39ab934092914ef3e4bb730b978"} err="failed to get container status \"f0b0a4a477518187319a77324e173d051a53f39ab934092914ef3e4bb730b978\": rpc error: code = NotFound desc = could not find container \"f0b0a4a477518187319a77324e173d051a53f39ab934092914ef3e4bb730b978\": container with ID starting with f0b0a4a477518187319a77324e173d051a53f39ab934092914ef3e4bb730b978 not found: ID does not exist"
Jul 07 12:00:49.607224 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:49.607191    2570 scope.go:117] "RemoveContainer" containerID="133f0a0f2b936a9f6adba2f179d60a2a99b9c600c4d4b26fbbbee60a84c3dddd"
Jul 07 12:00:49.607474 ip-10-0-135-78 kubenswrapper[2570]: E0707 12:00:49.607460    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"133f0a0f2b936a9f6adba2f179d60a2a99b9c600c4d4b26fbbbee60a84c3dddd\": container with ID starting with 133f0a0f2b936a9f6adba2f179d60a2a99b9c600c4d4b26fbbbee60a84c3dddd not found: ID does not exist" containerID="133f0a0f2b936a9f6adba2f179d60a2a99b9c600c4d4b26fbbbee60a84c3dddd"
Jul 07 12:00:49.607520 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:49.607478    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"133f0a0f2b936a9f6adba2f179d60a2a99b9c600c4d4b26fbbbee60a84c3dddd"} err="failed to get container status \"133f0a0f2b936a9f6adba2f179d60a2a99b9c600c4d4b26fbbbee60a84c3dddd\": rpc error: code = NotFound desc = could not find container \"133f0a0f2b936a9f6adba2f179d60a2a99b9c600c4d4b26fbbbee60a84c3dddd\": container with ID starting with 133f0a0f2b936a9f6adba2f179d60a2a99b9c600c4d4b26fbbbee60a84c3dddd not found: ID does not exist"
Jul 07 12:00:49.747410 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:49.747363    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/53a5831a-7072-4369-9820-12aa1886dd84-workload-certs\") pod \"53a5831a-7072-4369-9820-12aa1886dd84\" (UID: \"53a5831a-7072-4369-9820-12aa1886dd84\") "
Jul 07 12:00:49.747410 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:49.747422    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dhdqh\" (UniqueName: \"kubernetes.io/projected/53a5831a-7072-4369-9820-12aa1886dd84-kube-api-access-dhdqh\") pod \"53a5831a-7072-4369-9820-12aa1886dd84\" (UID: \"53a5831a-7072-4369-9820-12aa1886dd84\") "
Jul 07 12:00:49.747677 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:49.747475    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/53a5831a-7072-4369-9820-12aa1886dd84-workload-socket\") pod \"53a5831a-7072-4369-9820-12aa1886dd84\" (UID: \"53a5831a-7072-4369-9820-12aa1886dd84\") "
Jul 07 12:00:49.747677 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:49.747534    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/53a5831a-7072-4369-9820-12aa1886dd84-istio-data\") pod \"53a5831a-7072-4369-9820-12aa1886dd84\" (UID: \"53a5831a-7072-4369-9820-12aa1886dd84\") "
Jul 07 12:00:49.747677 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:49.747563    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/53a5831a-7072-4369-9820-12aa1886dd84-istiod-ca-cert\") pod \"53a5831a-7072-4369-9820-12aa1886dd84\" (UID: \"53a5831a-7072-4369-9820-12aa1886dd84\") "
Jul 07 12:00:49.747677 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:49.747597    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/53a5831a-7072-4369-9820-12aa1886dd84-istio-token\") pod \"53a5831a-7072-4369-9820-12aa1886dd84\" (UID: \"53a5831a-7072-4369-9820-12aa1886dd84\") "
Jul 07 12:00:49.747677 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:49.747630    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/53a5831a-7072-4369-9820-12aa1886dd84-credential-socket\") pod \"53a5831a-7072-4369-9820-12aa1886dd84\" (UID: \"53a5831a-7072-4369-9820-12aa1886dd84\") "
Jul 07 12:00:49.747677 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:49.747674    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/53a5831a-7072-4369-9820-12aa1886dd84-istio-podinfo\") pod \"53a5831a-7072-4369-9820-12aa1886dd84\" (UID: \"53a5831a-7072-4369-9820-12aa1886dd84\") "
Jul 07 12:00:49.747996 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:49.747697    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/53a5831a-7072-4369-9820-12aa1886dd84-istio-envoy\") pod \"53a5831a-7072-4369-9820-12aa1886dd84\" (UID: \"53a5831a-7072-4369-9820-12aa1886dd84\") "
Jul 07 12:00:49.747996 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:49.747764    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53a5831a-7072-4369-9820-12aa1886dd84-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "53a5831a-7072-4369-9820-12aa1886dd84" (UID: "53a5831a-7072-4369-9820-12aa1886dd84"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:00:49.747996 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:49.747820    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53a5831a-7072-4369-9820-12aa1886dd84-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "53a5831a-7072-4369-9820-12aa1886dd84" (UID: "53a5831a-7072-4369-9820-12aa1886dd84"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:00:49.748156 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:49.748010    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53a5831a-7072-4369-9820-12aa1886dd84-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "53a5831a-7072-4369-9820-12aa1886dd84" (UID: "53a5831a-7072-4369-9820-12aa1886dd84"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 12:00:49.748156 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:49.748038    2570 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/53a5831a-7072-4369-9820-12aa1886dd84-workload-certs\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:00:49.748156 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:49.748059    2570 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/53a5831a-7072-4369-9820-12aa1886dd84-workload-socket\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:00:49.748349 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:49.748316    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53a5831a-7072-4369-9820-12aa1886dd84-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "53a5831a-7072-4369-9820-12aa1886dd84" (UID: "53a5831a-7072-4369-9820-12aa1886dd84"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:00:49.748600 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:49.748577    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53a5831a-7072-4369-9820-12aa1886dd84-istio-data" (OuterVolumeSpecName: "istio-data") pod "53a5831a-7072-4369-9820-12aa1886dd84" (UID: "53a5831a-7072-4369-9820-12aa1886dd84"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:00:49.749900 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:49.749870    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53a5831a-7072-4369-9820-12aa1886dd84-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "53a5831a-7072-4369-9820-12aa1886dd84" (UID: "53a5831a-7072-4369-9820-12aa1886dd84"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:00:49.750008 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:49.749934    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53a5831a-7072-4369-9820-12aa1886dd84-istio-token" (OuterVolumeSpecName: "istio-token") pod "53a5831a-7072-4369-9820-12aa1886dd84" (UID: "53a5831a-7072-4369-9820-12aa1886dd84"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 12:00:49.750094 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:49.750073    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53a5831a-7072-4369-9820-12aa1886dd84-kube-api-access-dhdqh" (OuterVolumeSpecName: "kube-api-access-dhdqh") pod "53a5831a-7072-4369-9820-12aa1886dd84" (UID: "53a5831a-7072-4369-9820-12aa1886dd84"). InnerVolumeSpecName "kube-api-access-dhdqh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 12:00:49.750166 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:49.750151    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/53a5831a-7072-4369-9820-12aa1886dd84-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "53a5831a-7072-4369-9820-12aa1886dd84" (UID: "53a5831a-7072-4369-9820-12aa1886dd84"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 12:00:49.848609 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:49.848522    2570 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/53a5831a-7072-4369-9820-12aa1886dd84-credential-socket\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:00:49.848609 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:49.848556    2570 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/53a5831a-7072-4369-9820-12aa1886dd84-istio-podinfo\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:00:49.848609 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:49.848565    2570 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/53a5831a-7072-4369-9820-12aa1886dd84-istio-envoy\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:00:49.848609 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:49.848573    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dhdqh\" (UniqueName: \"kubernetes.io/projected/53a5831a-7072-4369-9820-12aa1886dd84-kube-api-access-dhdqh\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:00:49.848609 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:49.848583    2570 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/53a5831a-7072-4369-9820-12aa1886dd84-istio-data\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:00:49.848609 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:49.848590    2570 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/53a5831a-7072-4369-9820-12aa1886dd84-istiod-ca-cert\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:00:49.848609 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:49.848598    2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/53a5831a-7072-4369-9820-12aa1886dd84-istio-token\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:00:49.915875 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:49.915843    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9"]
Jul 07 12:00:49.919603 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:49.919577    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-b80bcdc6-openshift-default-6f88df9d79-7dkw9"]
Jul 07 12:00:49.938591 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:00:49.938558    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53a5831a-7072-4369-9820-12aa1886dd84" path="/var/lib/kubelet/pods/53a5831a-7072-4369-9820-12aa1886dd84/volumes"
Jul 07 12:01:49.285864 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:01:49.285823    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt"]
Jul 07 12:01:49.286374 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:01:49.286358    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0620ae1-01ac-437e-bcb2-9f85957e0abc" containerName="main"
Jul 07 12:01:49.286425 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:01:49.286377    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0620ae1-01ac-437e-bcb2-9f85957e0abc" containerName="main"
Jul 07 12:01:49.286425 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:01:49.286391    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53a5831a-7072-4369-9820-12aa1886dd84" containerName="istio-proxy"
Jul 07 12:01:49.286425 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:01:49.286399    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="53a5831a-7072-4369-9820-12aa1886dd84" containerName="istio-proxy"
Jul 07 12:01:49.286526 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:01:49.286485    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="53a5831a-7072-4369-9820-12aa1886dd84" containerName="istio-proxy"
Jul 07 12:01:49.286526 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:01:49.286501    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0620ae1-01ac-437e-bcb2-9f85957e0abc" containerName="main"
Jul 07 12:01:49.289951 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:01:49.289930    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt"
Jul 07 12:01:49.292966 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:01:49.292942    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-02e907-openshift-default-dockercfg-jntcp\""
Jul 07 12:01:49.302000 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:01:49.301975    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt"]
Jul 07 12:01:49.378986 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:01:49.378952    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72cd0119-5c4d-459a-938b-a900543eee7d-istiod-ca-cert\") pod \"e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt\" (UID: \"72cd0119-5c4d-459a-938b-a900543eee7d\") " pod="openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt"
Jul 07 12:01:49.378986 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:01:49.378994    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72cd0119-5c4d-459a-938b-a900543eee7d-credential-socket\") pod \"e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt\" (UID: \"72cd0119-5c4d-459a-938b-a900543eee7d\") " pod="openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt"
Jul 07 12:01:49.379294 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:01:49.379013    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72cd0119-5c4d-459a-938b-a900543eee7d-istio-token\") pod \"e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt\" (UID: \"72cd0119-5c4d-459a-938b-a900543eee7d\") " pod="openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt"
Jul 07 12:01:49.379294 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:01:49.379092    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72cd0119-5c4d-459a-938b-a900543eee7d-istio-podinfo\") pod \"e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt\" (UID: \"72cd0119-5c4d-459a-938b-a900543eee7d\") " pod="openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt"
Jul 07 12:01:49.379294 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:01:49.379162    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72cd0119-5c4d-459a-938b-a900543eee7d-istio-data\") pod \"e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt\" (UID: \"72cd0119-5c4d-459a-938b-a900543eee7d\") " pod="openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt"
Jul 07 12:01:49.379294 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:01:49.379181    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtmcl\" (UniqueName: \"kubernetes.io/projected/72cd0119-5c4d-459a-938b-a900543eee7d-kube-api-access-xtmcl\") pod \"e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt\" (UID: \"72cd0119-5c4d-459a-938b-a900543eee7d\") " pod="openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt"
Jul 07 12:01:49.379294 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:01:49.379250    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72cd0119-5c4d-459a-938b-a900543eee7d-istio-envoy\") pod \"e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt\" (UID: \"72cd0119-5c4d-459a-938b-a900543eee7d\") " pod="openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt"
Jul 07 12:01:49.379572 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:01:49.379326    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72cd0119-5c4d-459a-938b-a900543eee7d-workload-certs\") pod \"e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt\" (UID: \"72cd0119-5c4d-459a-938b-a900543eee7d\") " pod="openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt"
Jul 07 12:01:49.379572 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:01:49.379355    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72cd0119-5c4d-459a-938b-a900543eee7d-workload-socket\") pod \"e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt\" (UID: \"72cd0119-5c4d-459a-938b-a900543eee7d\") " pod="openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt"
Jul 07 12:01:49.480649 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:01:49.480609    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72cd0119-5c4d-459a-938b-a900543eee7d-istiod-ca-cert\") pod \"e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt\" (UID: \"72cd0119-5c4d-459a-938b-a900543eee7d\") " pod="openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt"
Jul 07 12:01:49.480649 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:01:49.480651    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72cd0119-5c4d-459a-938b-a900543eee7d-credential-socket\") pod \"e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt\" (UID: \"72cd0119-5c4d-459a-938b-a900543eee7d\") " pod="openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt"
Jul 07 12:01:49.480927 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:01:49.480675    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72cd0119-5c4d-459a-938b-a900543eee7d-istio-token\") pod \"e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt\" (UID: \"72cd0119-5c4d-459a-938b-a900543eee7d\") " pod="openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt"
Jul 07 12:01:49.480927 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:01:49.480720    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72cd0119-5c4d-459a-938b-a900543eee7d-istio-podinfo\") pod \"e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt\" (UID: \"72cd0119-5c4d-459a-938b-a900543eee7d\") " pod="openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt"
Jul 07 12:01:49.480927 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:01:49.480751    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72cd0119-5c4d-459a-938b-a900543eee7d-istio-data\") pod \"e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt\" (UID: \"72cd0119-5c4d-459a-938b-a900543eee7d\") " pod="openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt"
Jul 07 12:01:49.480927 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:01:49.480782    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtmcl\" (UniqueName: \"kubernetes.io/projected/72cd0119-5c4d-459a-938b-a900543eee7d-kube-api-access-xtmcl\") pod \"e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt\" (UID: \"72cd0119-5c4d-459a-938b-a900543eee7d\") " pod="openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt"
Jul 07 12:01:49.480927 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:01:49.480809    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72cd0119-5c4d-459a-938b-a900543eee7d-istio-envoy\") pod \"e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt\" (UID: \"72cd0119-5c4d-459a-938b-a900543eee7d\") " pod="openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt"
Jul 07 12:01:49.480927 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:01:49.480864    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72cd0119-5c4d-459a-938b-a900543eee7d-workload-certs\") pod \"e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt\" (UID: \"72cd0119-5c4d-459a-938b-a900543eee7d\") " pod="openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt"
Jul 07 12:01:49.480927 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:01:49.480895    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72cd0119-5c4d-459a-938b-a900543eee7d-workload-socket\") pod \"e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt\" (UID: \"72cd0119-5c4d-459a-938b-a900543eee7d\") " pod="openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt"
Jul 07 12:01:49.481427 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:01:49.481400    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72cd0119-5c4d-459a-938b-a900543eee7d-credential-socket\") pod \"e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt\" (UID: \"72cd0119-5c4d-459a-938b-a900543eee7d\") " pod="openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt"
Jul 07 12:01:49.481494 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:01:49.481444    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72cd0119-5c4d-459a-938b-a900543eee7d-istio-data\") pod \"e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt\" (UID: \"72cd0119-5c4d-459a-938b-a900543eee7d\") " pod="openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt"
Jul 07 12:01:49.481494 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:01:49.481470    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72cd0119-5c4d-459a-938b-a900543eee7d-workload-socket\") pod \"e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt\" (UID: \"72cd0119-5c4d-459a-938b-a900543eee7d\") " pod="openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt"
Jul 07 12:01:49.481494 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:01:49.481444    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72cd0119-5c4d-459a-938b-a900543eee7d-istiod-ca-cert\") pod \"e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt\" (UID: \"72cd0119-5c4d-459a-938b-a900543eee7d\") " pod="openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt"
Jul 07 12:01:49.481662 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:01:49.481634    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72cd0119-5c4d-459a-938b-a900543eee7d-workload-certs\") pod \"e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt\" (UID: \"72cd0119-5c4d-459a-938b-a900543eee7d\") " pod="openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt"
Jul 07 12:01:49.483455 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:01:49.483430    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72cd0119-5c4d-459a-938b-a900543eee7d-istio-envoy\") pod \"e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt\" (UID: \"72cd0119-5c4d-459a-938b-a900543eee7d\") " pod="openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt"
Jul 07 12:01:49.483622 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:01:49.483603    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72cd0119-5c4d-459a-938b-a900543eee7d-istio-podinfo\") pod \"e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt\" (UID: \"72cd0119-5c4d-459a-938b-a900543eee7d\") " pod="openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt"
Jul 07 12:01:49.490336 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:01:49.490313    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72cd0119-5c4d-459a-938b-a900543eee7d-istio-token\") pod \"e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt\" (UID: \"72cd0119-5c4d-459a-938b-a900543eee7d\") " pod="openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt"
Jul 07 12:01:49.490467 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:01:49.490448    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtmcl\" (UniqueName: \"kubernetes.io/projected/72cd0119-5c4d-459a-938b-a900543eee7d-kube-api-access-xtmcl\") pod \"e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt\" (UID: \"72cd0119-5c4d-459a-938b-a900543eee7d\") " pod="openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt"
Jul 07 12:01:49.602144 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:01:49.602046    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt"
Jul 07 12:01:49.755495 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:01:49.755408    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt"]
Jul 07 12:01:49.756777 ip-10-0-135-78 kubenswrapper[2570]: W0707 12:01:49.756747    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72cd0119_5c4d_459a_938b_a900543eee7d.slice/crio-e043b2754efeef616638aa7eab784272982c34dfa739002793daa94f0fe4b010 WatchSource:0}: Error finding container e043b2754efeef616638aa7eab784272982c34dfa739002793daa94f0fe4b010: Status 404 returned error can't find the container with id e043b2754efeef616638aa7eab784272982c34dfa739002793daa94f0fe4b010
Jul 07 12:01:49.758922 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:01:49.758887    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 12:01:49.759002 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:01:49.758958    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 12:01:49.759002 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:01:49.758987    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 12:01:49.903433 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:01:49.903325    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt" event={"ID":"72cd0119-5c4d-459a-938b-a900543eee7d","Type":"ContainerStarted","Data":"f9a5e5b6098d67e334d03cd7e8b5aebefb4160a0d49ca54460cf1c598093cec5"}
Jul 07 12:01:49.903433 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:01:49.903389    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt" event={"ID":"72cd0119-5c4d-459a-938b-a900543eee7d","Type":"ContainerStarted","Data":"e043b2754efeef616638aa7eab784272982c34dfa739002793daa94f0fe4b010"}
Jul 07 12:01:49.923581 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:01:49.923512    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt" podStartSLOduration=0.923490691 podStartE2EDuration="923.490691ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:01:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:01:49.923423748 +0000 UTC m=+2097.897064166" watchObservedRunningTime="2026-07-07 12:01:49.923490691 +0000 UTC m=+2097.897131103"
Jul 07 12:01:50.602420 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:01:50.602380    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt"
Jul 07 12:01:50.607256 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:01:50.607229    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt"
Jul 07 12:01:50.909186 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:01:50.909087    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt"
Jul 07 12:01:50.910299 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:01:50.910274    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt"
Jul 07 12:01:54.035913 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:01:54.035883    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ktbpw_faf6f343-6515-4b2c-a34e-286cd2d9ec7a/console-operator/1.log"
Jul 07 12:01:54.044532 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:01:54.044507    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ktbpw_faf6f343-6515-4b2c-a34e-286cd2d9ec7a/console-operator/1.log"
Jul 07 12:02:01.816139 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:01.816108    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-302b88/test-model-302b88-kserve-7b8f66cc6c-vdh88"]
Jul 07 12:02:01.816621 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:01.816518    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53a5831a-7072-4369-9820-12aa1886dd84" containerName="istio-proxy"
Jul 07 12:02:01.816621 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:01.816531    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="53a5831a-7072-4369-9820-12aa1886dd84" containerName="istio-proxy"
Jul 07 12:02:01.816621 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:01.816597    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="53a5831a-7072-4369-9820-12aa1886dd84" containerName="istio-proxy"
Jul 07 12:02:01.819780 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:01.819758    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-302b88/test-model-302b88-kserve-7b8f66cc6c-vdh88"
Jul 07 12:02:01.824417 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:01.824393    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-302b88\"/\"test-model-302b88-kserve-self-signed-certs\""
Jul 07 12:02:01.824417 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:01.824415    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-302b88\"/\"openshift-service-ca.crt\""
Jul 07 12:02:01.824631 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:01.824393    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-302b88\"/\"default-dockercfg-wg45x\""
Jul 07 12:02:01.824631 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:01.824395    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-302b88\"/\"kube-root-ca.crt\""
Jul 07 12:02:01.831170 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:01.831144    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-302b88/test-model-302b88-kserve-7b8f66cc6c-vdh88"]
Jul 07 12:02:01.896890 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:01.896847    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ccb093dd-bfe8-43a6-b0be-3470585b2456-dshm\") pod \"test-model-302b88-kserve-7b8f66cc6c-vdh88\" (UID: \"ccb093dd-bfe8-43a6-b0be-3470585b2456\") " pod="ai-tenant-e2e-inf-a-302b88/test-model-302b88-kserve-7b8f66cc6c-vdh88"
Jul 07 12:02:01.897079 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:01.896946    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nrpk\" (UniqueName: \"kubernetes.io/projected/ccb093dd-bfe8-43a6-b0be-3470585b2456-kube-api-access-8nrpk\") pod \"test-model-302b88-kserve-7b8f66cc6c-vdh88\" (UID: \"ccb093dd-bfe8-43a6-b0be-3470585b2456\") " pod="ai-tenant-e2e-inf-a-302b88/test-model-302b88-kserve-7b8f66cc6c-vdh88"
Jul 07 12:02:01.897079 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:01.896979    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ccb093dd-bfe8-43a6-b0be-3470585b2456-model-cache\") pod \"test-model-302b88-kserve-7b8f66cc6c-vdh88\" (UID: \"ccb093dd-bfe8-43a6-b0be-3470585b2456\") " pod="ai-tenant-e2e-inf-a-302b88/test-model-302b88-kserve-7b8f66cc6c-vdh88"
Jul 07 12:02:01.897079 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:01.897016    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ccb093dd-bfe8-43a6-b0be-3470585b2456-tmp-dir\") pod \"test-model-302b88-kserve-7b8f66cc6c-vdh88\" (UID: \"ccb093dd-bfe8-43a6-b0be-3470585b2456\") " pod="ai-tenant-e2e-inf-a-302b88/test-model-302b88-kserve-7b8f66cc6c-vdh88"
Jul 07 12:02:01.897079 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:01.897041    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ccb093dd-bfe8-43a6-b0be-3470585b2456-home\") pod \"test-model-302b88-kserve-7b8f66cc6c-vdh88\" (UID: \"ccb093dd-bfe8-43a6-b0be-3470585b2456\") " pod="ai-tenant-e2e-inf-a-302b88/test-model-302b88-kserve-7b8f66cc6c-vdh88"
Jul 07 12:02:01.897262 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:01.897096    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ccb093dd-bfe8-43a6-b0be-3470585b2456-tls-certs\") pod \"test-model-302b88-kserve-7b8f66cc6c-vdh88\" (UID: \"ccb093dd-bfe8-43a6-b0be-3470585b2456\") " pod="ai-tenant-e2e-inf-a-302b88/test-model-302b88-kserve-7b8f66cc6c-vdh88"
Jul 07 12:02:01.998069 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:01.998032    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nrpk\" (UniqueName: \"kubernetes.io/projected/ccb093dd-bfe8-43a6-b0be-3470585b2456-kube-api-access-8nrpk\") pod \"test-model-302b88-kserve-7b8f66cc6c-vdh88\" (UID: \"ccb093dd-bfe8-43a6-b0be-3470585b2456\") " pod="ai-tenant-e2e-inf-a-302b88/test-model-302b88-kserve-7b8f66cc6c-vdh88"
Jul 07 12:02:01.998286 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:01.998078    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ccb093dd-bfe8-43a6-b0be-3470585b2456-model-cache\") pod \"test-model-302b88-kserve-7b8f66cc6c-vdh88\" (UID: \"ccb093dd-bfe8-43a6-b0be-3470585b2456\") " pod="ai-tenant-e2e-inf-a-302b88/test-model-302b88-kserve-7b8f66cc6c-vdh88"
Jul 07 12:02:01.998286 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:01.998096    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ccb093dd-bfe8-43a6-b0be-3470585b2456-tmp-dir\") pod \"test-model-302b88-kserve-7b8f66cc6c-vdh88\" (UID: \"ccb093dd-bfe8-43a6-b0be-3470585b2456\") " pod="ai-tenant-e2e-inf-a-302b88/test-model-302b88-kserve-7b8f66cc6c-vdh88"
Jul 07 12:02:01.998286 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:01.998249    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ccb093dd-bfe8-43a6-b0be-3470585b2456-home\") pod \"test-model-302b88-kserve-7b8f66cc6c-vdh88\" (UID: \"ccb093dd-bfe8-43a6-b0be-3470585b2456\") " pod="ai-tenant-e2e-inf-a-302b88/test-model-302b88-kserve-7b8f66cc6c-vdh88"
Jul 07 12:02:01.998469 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:01.998302    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ccb093dd-bfe8-43a6-b0be-3470585b2456-tls-certs\") pod \"test-model-302b88-kserve-7b8f66cc6c-vdh88\" (UID: \"ccb093dd-bfe8-43a6-b0be-3470585b2456\") " pod="ai-tenant-e2e-inf-a-302b88/test-model-302b88-kserve-7b8f66cc6c-vdh88"
Jul 07 12:02:01.998469 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:01.998408    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ccb093dd-bfe8-43a6-b0be-3470585b2456-dshm\") pod \"test-model-302b88-kserve-7b8f66cc6c-vdh88\" (UID: \"ccb093dd-bfe8-43a6-b0be-3470585b2456\") " pod="ai-tenant-e2e-inf-a-302b88/test-model-302b88-kserve-7b8f66cc6c-vdh88"
Jul 07 12:02:01.998581 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:01.998467    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ccb093dd-bfe8-43a6-b0be-3470585b2456-tmp-dir\") pod \"test-model-302b88-kserve-7b8f66cc6c-vdh88\" (UID: \"ccb093dd-bfe8-43a6-b0be-3470585b2456\") " pod="ai-tenant-e2e-inf-a-302b88/test-model-302b88-kserve-7b8f66cc6c-vdh88"
Jul 07 12:02:01.998581 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:01.998541    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ccb093dd-bfe8-43a6-b0be-3470585b2456-model-cache\") pod \"test-model-302b88-kserve-7b8f66cc6c-vdh88\" (UID: \"ccb093dd-bfe8-43a6-b0be-3470585b2456\") " pod="ai-tenant-e2e-inf-a-302b88/test-model-302b88-kserve-7b8f66cc6c-vdh88"
Jul 07 12:02:01.998690 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:01.998636    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ccb093dd-bfe8-43a6-b0be-3470585b2456-home\") pod \"test-model-302b88-kserve-7b8f66cc6c-vdh88\" (UID: \"ccb093dd-bfe8-43a6-b0be-3470585b2456\") " pod="ai-tenant-e2e-inf-a-302b88/test-model-302b88-kserve-7b8f66cc6c-vdh88"
Jul 07 12:02:02.000553 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:02.000520    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ccb093dd-bfe8-43a6-b0be-3470585b2456-dshm\") pod \"test-model-302b88-kserve-7b8f66cc6c-vdh88\" (UID: \"ccb093dd-bfe8-43a6-b0be-3470585b2456\") " pod="ai-tenant-e2e-inf-a-302b88/test-model-302b88-kserve-7b8f66cc6c-vdh88"
Jul 07 12:02:02.000938 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:02.000919    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ccb093dd-bfe8-43a6-b0be-3470585b2456-tls-certs\") pod \"test-model-302b88-kserve-7b8f66cc6c-vdh88\" (UID: \"ccb093dd-bfe8-43a6-b0be-3470585b2456\") " pod="ai-tenant-e2e-inf-a-302b88/test-model-302b88-kserve-7b8f66cc6c-vdh88"
Jul 07 12:02:02.007536 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:02.007518    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nrpk\" (UniqueName: \"kubernetes.io/projected/ccb093dd-bfe8-43a6-b0be-3470585b2456-kube-api-access-8nrpk\") pod \"test-model-302b88-kserve-7b8f66cc6c-vdh88\" (UID: \"ccb093dd-bfe8-43a6-b0be-3470585b2456\") " pod="ai-tenant-e2e-inf-a-302b88/test-model-302b88-kserve-7b8f66cc6c-vdh88"
Jul 07 12:02:02.130487 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:02.130391    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-302b88/test-model-302b88-kserve-7b8f66cc6c-vdh88"
Jul 07 12:02:02.270667 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:02.270630    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-302b88/test-model-302b88-kserve-7b8f66cc6c-vdh88"]
Jul 07 12:02:02.272301 ip-10-0-135-78 kubenswrapper[2570]: W0707 12:02:02.272274    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podccb093dd_bfe8_43a6_b0be_3470585b2456.slice/crio-724c322f479e560943ca4e76224f9820735c1745c48d7f44d525c52a1615b8aa WatchSource:0}: Error finding container 724c322f479e560943ca4e76224f9820735c1745c48d7f44d525c52a1615b8aa: Status 404 returned error can't find the container with id 724c322f479e560943ca4e76224f9820735c1745c48d7f44d525c52a1615b8aa
Jul 07 12:02:02.975505 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:02.975465    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-302b88/test-model-302b88-kserve-7b8f66cc6c-vdh88" event={"ID":"ccb093dd-bfe8-43a6-b0be-3470585b2456","Type":"ContainerStarted","Data":"1d1a3320f13ef33e576648aca9d08279968acf9167dfab550d64c46c58f63cff"}
Jul 07 12:02:02.975505 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:02.975512    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-302b88/test-model-302b88-kserve-7b8f66cc6c-vdh88" event={"ID":"ccb093dd-bfe8-43a6-b0be-3470585b2456","Type":"ContainerStarted","Data":"724c322f479e560943ca4e76224f9820735c1745c48d7f44d525c52a1615b8aa"}
Jul 07 12:02:02.993732 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:02.993680    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-302b88/test-model-302b88-kserve-7b8f66cc6c-vdh88" podStartSLOduration=1.9936634469999999 podStartE2EDuration="1.993663447s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:02:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:02:02.992953052 +0000 UTC m=+2110.966593480" watchObservedRunningTime="2026-07-07 12:02:02.993663447 +0000 UTC m=+2110.967303857"
Jul 07 12:02:10.014916 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:10.014881    2570 generic.go:356] "Generic (PLEG): container finished" podID="72cd0119-5c4d-459a-938b-a900543eee7d" containerID="f9a5e5b6098d67e334d03cd7e8b5aebefb4160a0d49ca54460cf1c598093cec5" exitCode=0
Jul 07 12:02:10.015382 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:10.014964    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt" event={"ID":"72cd0119-5c4d-459a-938b-a900543eee7d","Type":"ContainerDied","Data":"f9a5e5b6098d67e334d03cd7e8b5aebefb4160a0d49ca54460cf1c598093cec5"}
Jul 07 12:02:10.015471 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:10.015454    2570 scope.go:117] "RemoveContainer" containerID="f9a5e5b6098d67e334d03cd7e8b5aebefb4160a0d49ca54460cf1c598093cec5"
Jul 07 12:02:10.016486 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:10.016458    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 12:02:10.016604 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:10.016562    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 12:02:10.016669 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:10.016614    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 12:02:11.024581 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:11.024528    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt" event={"ID":"72cd0119-5c4d-459a-938b-a900543eee7d","Type":"ContainerStarted","Data":"30ea07dac0c7ffef9e6fa840e413a7a11a06b5375cb0792f7e3cc247ffe312b0"}
Jul 07 12:02:11.602475 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:11.602424    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt"
Jul 07 12:02:11.602757 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:11.602731    2570 patch_prober.go:28] interesting pod/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 07 12:02:11.602862 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:11.602802    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt" podUID="72cd0119-5c4d-459a-938b-a900543eee7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 07 12:02:12.133238 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:12.131388    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-302b88/test-model-302b88-kserve-7b8f66cc6c-vdh88"
Jul 07 12:02:12.137233 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:12.134108    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-302b88/test-model-302b88-kserve-7b8f66cc6c-vdh88"
Jul 07 12:02:12.156738 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:12.156710    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-302b88/test-model-302b88-kserve-7b8f66cc6c-vdh88"
Jul 07 12:02:12.602546 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:12.602508    2570 patch_prober.go:28] interesting pod/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 07 12:02:12.602753 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:12.602578    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt" podUID="72cd0119-5c4d-459a-938b-a900543eee7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 07 12:02:13.056351 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:13.056315    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-302b88/test-model-302b88-kserve-7b8f66cc6c-vdh88"
Jul 07 12:02:13.602995 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:13.602962    2570 patch_prober.go:28] interesting pod/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 07 12:02:13.603427 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:13.603012    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt" podUID="72cd0119-5c4d-459a-938b-a900543eee7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 07 12:02:14.602475 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:14.602441    2570 patch_prober.go:28] interesting pod/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 07 12:02:14.602676 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:14.602498    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt" podUID="72cd0119-5c4d-459a-938b-a900543eee7d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 07 12:02:15.606561 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:15.606523    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt"
Jul 07 12:02:15.607030 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:15.606801    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt"
Jul 07 12:02:15.607628 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:15.607610    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt"
Jul 07 12:02:22.729885 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:22.729854    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-02e907/test-model-02e907-kserve-bbb89ff55-4s2zl"]
Jul 07 12:02:22.732615 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:22.732598    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-02e907/test-model-02e907-kserve-bbb89ff55-4s2zl"
Jul 07 12:02:22.735386 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:22.735359    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-02e907\"/\"openshift-service-ca.crt\""
Jul 07 12:02:22.735498 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:22.735438    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-02e907\"/\"kube-root-ca.crt\""
Jul 07 12:02:22.736628 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:22.736592    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-02e907\"/\"default-dockercfg-rrkd6\""
Jul 07 12:02:22.736743 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:22.736687    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-02e907\"/\"test-model-02e907-kserve-self-signed-certs\""
Jul 07 12:02:22.743697 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:22.743675    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-02e907/test-model-02e907-kserve-bbb89ff55-4s2zl"]
Jul 07 12:02:22.911026 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:22.910982    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwmkl\" (UniqueName: \"kubernetes.io/projected/865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1-kube-api-access-rwmkl\") pod \"test-model-02e907-kserve-bbb89ff55-4s2zl\" (UID: \"865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1\") " pod="ai-tenant-e2e-inf-b-02e907/test-model-02e907-kserve-bbb89ff55-4s2zl"
Jul 07 12:02:22.911026 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:22.911036    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1-tmp-dir\") pod \"test-model-02e907-kserve-bbb89ff55-4s2zl\" (UID: \"865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1\") " pod="ai-tenant-e2e-inf-b-02e907/test-model-02e907-kserve-bbb89ff55-4s2zl"
Jul 07 12:02:22.911380 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:22.911149    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1-model-cache\") pod \"test-model-02e907-kserve-bbb89ff55-4s2zl\" (UID: \"865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1\") " pod="ai-tenant-e2e-inf-b-02e907/test-model-02e907-kserve-bbb89ff55-4s2zl"
Jul 07 12:02:22.911380 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:22.911241    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1-tls-certs\") pod \"test-model-02e907-kserve-bbb89ff55-4s2zl\" (UID: \"865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1\") " pod="ai-tenant-e2e-inf-b-02e907/test-model-02e907-kserve-bbb89ff55-4s2zl"
Jul 07 12:02:22.911380 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:22.911274    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1-dshm\") pod \"test-model-02e907-kserve-bbb89ff55-4s2zl\" (UID: \"865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1\") " pod="ai-tenant-e2e-inf-b-02e907/test-model-02e907-kserve-bbb89ff55-4s2zl"
Jul 07 12:02:22.911380 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:22.911352    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1-home\") pod \"test-model-02e907-kserve-bbb89ff55-4s2zl\" (UID: \"865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1\") " pod="ai-tenant-e2e-inf-b-02e907/test-model-02e907-kserve-bbb89ff55-4s2zl"
Jul 07 12:02:23.012827 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:23.012726    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1-home\") pod \"test-model-02e907-kserve-bbb89ff55-4s2zl\" (UID: \"865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1\") " pod="ai-tenant-e2e-inf-b-02e907/test-model-02e907-kserve-bbb89ff55-4s2zl"
Jul 07 12:02:23.012827 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:23.012820    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwmkl\" (UniqueName: \"kubernetes.io/projected/865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1-kube-api-access-rwmkl\") pod \"test-model-02e907-kserve-bbb89ff55-4s2zl\" (UID: \"865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1\") " pod="ai-tenant-e2e-inf-b-02e907/test-model-02e907-kserve-bbb89ff55-4s2zl"
Jul 07 12:02:23.013065 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:23.012859    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1-tmp-dir\") pod \"test-model-02e907-kserve-bbb89ff55-4s2zl\" (UID: \"865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1\") " pod="ai-tenant-e2e-inf-b-02e907/test-model-02e907-kserve-bbb89ff55-4s2zl"
Jul 07 12:02:23.013065 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:23.012896    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1-model-cache\") pod \"test-model-02e907-kserve-bbb89ff55-4s2zl\" (UID: \"865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1\") " pod="ai-tenant-e2e-inf-b-02e907/test-model-02e907-kserve-bbb89ff55-4s2zl"
Jul 07 12:02:23.013065 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:23.012942    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1-tls-certs\") pod \"test-model-02e907-kserve-bbb89ff55-4s2zl\" (UID: \"865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1\") " pod="ai-tenant-e2e-inf-b-02e907/test-model-02e907-kserve-bbb89ff55-4s2zl"
Jul 07 12:02:23.013065 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:23.012966    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1-dshm\") pod \"test-model-02e907-kserve-bbb89ff55-4s2zl\" (UID: \"865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1\") " pod="ai-tenant-e2e-inf-b-02e907/test-model-02e907-kserve-bbb89ff55-4s2zl"
Jul 07 12:02:23.013305 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:23.013244    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1-home\") pod \"test-model-02e907-kserve-bbb89ff55-4s2zl\" (UID: \"865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1\") " pod="ai-tenant-e2e-inf-b-02e907/test-model-02e907-kserve-bbb89ff55-4s2zl"
Jul 07 12:02:23.013348 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:23.013303    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1-tmp-dir\") pod \"test-model-02e907-kserve-bbb89ff55-4s2zl\" (UID: \"865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1\") " pod="ai-tenant-e2e-inf-b-02e907/test-model-02e907-kserve-bbb89ff55-4s2zl"
Jul 07 12:02:23.013402 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:23.013383    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1-model-cache\") pod \"test-model-02e907-kserve-bbb89ff55-4s2zl\" (UID: \"865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1\") " pod="ai-tenant-e2e-inf-b-02e907/test-model-02e907-kserve-bbb89ff55-4s2zl"
Jul 07 12:02:23.015291 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:23.015270    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1-dshm\") pod \"test-model-02e907-kserve-bbb89ff55-4s2zl\" (UID: \"865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1\") " pod="ai-tenant-e2e-inf-b-02e907/test-model-02e907-kserve-bbb89ff55-4s2zl"
Jul 07 12:02:23.015493 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:23.015475    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1-tls-certs\") pod \"test-model-02e907-kserve-bbb89ff55-4s2zl\" (UID: \"865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1\") " pod="ai-tenant-e2e-inf-b-02e907/test-model-02e907-kserve-bbb89ff55-4s2zl"
Jul 07 12:02:23.021533 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:23.021507    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwmkl\" (UniqueName: \"kubernetes.io/projected/865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1-kube-api-access-rwmkl\") pod \"test-model-02e907-kserve-bbb89ff55-4s2zl\" (UID: \"865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1\") " pod="ai-tenant-e2e-inf-b-02e907/test-model-02e907-kserve-bbb89ff55-4s2zl"
Jul 07 12:02:23.043061 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:23.043022    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-02e907/test-model-02e907-kserve-bbb89ff55-4s2zl"
Jul 07 12:02:23.188700 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:23.188667    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-02e907/test-model-02e907-kserve-bbb89ff55-4s2zl"]
Jul 07 12:02:23.190567 ip-10-0-135-78 kubenswrapper[2570]: W0707 12:02:23.190527    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod865b0cdd_0ab5_45da_bb32_0fc9e6f7b2d1.slice/crio-791651f46ecd14f8661eae720a5a77501327b7774ef814dee5dc46e559b1a02d WatchSource:0}: Error finding container 791651f46ecd14f8661eae720a5a77501327b7774ef814dee5dc46e559b1a02d: Status 404 returned error can't find the container with id 791651f46ecd14f8661eae720a5a77501327b7774ef814dee5dc46e559b1a02d
Jul 07 12:02:24.102999 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:24.102960    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-02e907/test-model-02e907-kserve-bbb89ff55-4s2zl" event={"ID":"865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1","Type":"ContainerStarted","Data":"f44b874331f4e92ae4ccfb26dcf6d6f36f17c968cc5a05e03dd155e513076c3f"}
Jul 07 12:02:24.102999 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:24.103006    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-02e907/test-model-02e907-kserve-bbb89ff55-4s2zl" event={"ID":"865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1","Type":"ContainerStarted","Data":"791651f46ecd14f8661eae720a5a77501327b7774ef814dee5dc46e559b1a02d"}
Jul 07 12:02:24.122982 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:24.122924    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-02e907/test-model-02e907-kserve-bbb89ff55-4s2zl" podStartSLOduration=2.122906897 podStartE2EDuration="2.122906897s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:02:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:02:24.121610618 +0000 UTC m=+2132.095251028" watchObservedRunningTime="2026-07-07 12:02:24.122906897 +0000 UTC m=+2132.096547307"
Jul 07 12:02:33.043640 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:33.043599    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-02e907/test-model-02e907-kserve-bbb89ff55-4s2zl"
Jul 07 12:02:33.044148 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:33.043656    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-02e907/test-model-02e907-kserve-bbb89ff55-4s2zl"
Jul 07 12:02:33.056762 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:33.056732    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-02e907/test-model-02e907-kserve-bbb89ff55-4s2zl"
Jul 07 12:02:33.160497 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:33.160468    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-02e907/test-model-02e907-kserve-bbb89ff55-4s2zl"
Jul 07 12:02:39.294954 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:39.294912    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-302b88/test-model-302b88-kserve-7b8f66cc6c-vdh88"]
Jul 07 12:02:40.264402 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:40.264366    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-02e907/test-model-02e907-kserve-bbb89ff55-4s2zl"]
Jul 07 12:02:54.317457 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:54.317348    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-302b88/test-model-302b88-kserve-7b8f66cc6c-vdh88" podUID="ccb093dd-bfe8-43a6-b0be-3470585b2456" containerName="main" containerID="cri-o://1d1a3320f13ef33e576648aca9d08279968acf9167dfab550d64c46c58f63cff" gracePeriod=45
Jul 07 12:02:54.576979 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:54.576909    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-302b88/test-model-302b88-kserve-7b8f66cc6c-vdh88"
Jul 07 12:02:54.694075 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:54.694039    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ccb093dd-bfe8-43a6-b0be-3470585b2456-tmp-dir\") pod \"ccb093dd-bfe8-43a6-b0be-3470585b2456\" (UID: \"ccb093dd-bfe8-43a6-b0be-3470585b2456\") "
Jul 07 12:02:54.694280 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:54.694088    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ccb093dd-bfe8-43a6-b0be-3470585b2456-home\") pod \"ccb093dd-bfe8-43a6-b0be-3470585b2456\" (UID: \"ccb093dd-bfe8-43a6-b0be-3470585b2456\") "
Jul 07 12:02:54.694280 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:54.694130    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8nrpk\" (UniqueName: \"kubernetes.io/projected/ccb093dd-bfe8-43a6-b0be-3470585b2456-kube-api-access-8nrpk\") pod \"ccb093dd-bfe8-43a6-b0be-3470585b2456\" (UID: \"ccb093dd-bfe8-43a6-b0be-3470585b2456\") "
Jul 07 12:02:54.694280 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:54.694158    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ccb093dd-bfe8-43a6-b0be-3470585b2456-tls-certs\") pod \"ccb093dd-bfe8-43a6-b0be-3470585b2456\" (UID: \"ccb093dd-bfe8-43a6-b0be-3470585b2456\") "
Jul 07 12:02:54.694280 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:54.694190    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ccb093dd-bfe8-43a6-b0be-3470585b2456-dshm\") pod \"ccb093dd-bfe8-43a6-b0be-3470585b2456\" (UID: \"ccb093dd-bfe8-43a6-b0be-3470585b2456\") "
Jul 07 12:02:54.694280 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:54.694261    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ccb093dd-bfe8-43a6-b0be-3470585b2456-model-cache\") pod \"ccb093dd-bfe8-43a6-b0be-3470585b2456\" (UID: \"ccb093dd-bfe8-43a6-b0be-3470585b2456\") "
Jul 07 12:02:54.694562 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:54.694411    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ccb093dd-bfe8-43a6-b0be-3470585b2456-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ccb093dd-bfe8-43a6-b0be-3470585b2456" (UID: "ccb093dd-bfe8-43a6-b0be-3470585b2456"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:02:54.694562 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:54.694448    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ccb093dd-bfe8-43a6-b0be-3470585b2456-home" (OuterVolumeSpecName: "home") pod "ccb093dd-bfe8-43a6-b0be-3470585b2456" (UID: "ccb093dd-bfe8-43a6-b0be-3470585b2456"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:02:54.694665 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:54.694556    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ccb093dd-bfe8-43a6-b0be-3470585b2456-model-cache" (OuterVolumeSpecName: "model-cache") pod "ccb093dd-bfe8-43a6-b0be-3470585b2456" (UID: "ccb093dd-bfe8-43a6-b0be-3470585b2456"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:02:54.694752 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:54.694727    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ccb093dd-bfe8-43a6-b0be-3470585b2456-tmp-dir\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:02:54.694752 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:54.694755    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ccb093dd-bfe8-43a6-b0be-3470585b2456-home\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:02:54.694898 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:54.694767    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ccb093dd-bfe8-43a6-b0be-3470585b2456-model-cache\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:02:54.696458 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:54.696429    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ccb093dd-bfe8-43a6-b0be-3470585b2456-dshm" (OuterVolumeSpecName: "dshm") pod "ccb093dd-bfe8-43a6-b0be-3470585b2456" (UID: "ccb093dd-bfe8-43a6-b0be-3470585b2456"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:02:54.696576 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:54.696452    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ccb093dd-bfe8-43a6-b0be-3470585b2456-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ccb093dd-bfe8-43a6-b0be-3470585b2456" (UID: "ccb093dd-bfe8-43a6-b0be-3470585b2456"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 12:02:54.696576 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:54.696493    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ccb093dd-bfe8-43a6-b0be-3470585b2456-kube-api-access-8nrpk" (OuterVolumeSpecName: "kube-api-access-8nrpk") pod "ccb093dd-bfe8-43a6-b0be-3470585b2456" (UID: "ccb093dd-bfe8-43a6-b0be-3470585b2456"). InnerVolumeSpecName "kube-api-access-8nrpk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 12:02:54.796026 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:54.795988    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8nrpk\" (UniqueName: \"kubernetes.io/projected/ccb093dd-bfe8-43a6-b0be-3470585b2456-kube-api-access-8nrpk\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:02:54.796026 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:54.796021    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ccb093dd-bfe8-43a6-b0be-3470585b2456-tls-certs\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:02:54.796026 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:54.796033    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ccb093dd-bfe8-43a6-b0be-3470585b2456-dshm\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:02:55.269697 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:55.269657    2570 generic.go:356] "Generic (PLEG): container finished" podID="ccb093dd-bfe8-43a6-b0be-3470585b2456" containerID="1d1a3320f13ef33e576648aca9d08279968acf9167dfab550d64c46c58f63cff" exitCode=0
Jul 07 12:02:55.269881 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:55.269707    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-302b88/test-model-302b88-kserve-7b8f66cc6c-vdh88"
Jul 07 12:02:55.269881 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:55.269750    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-302b88/test-model-302b88-kserve-7b8f66cc6c-vdh88" event={"ID":"ccb093dd-bfe8-43a6-b0be-3470585b2456","Type":"ContainerDied","Data":"1d1a3320f13ef33e576648aca9d08279968acf9167dfab550d64c46c58f63cff"}
Jul 07 12:02:55.269881 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:55.269799    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-302b88/test-model-302b88-kserve-7b8f66cc6c-vdh88" event={"ID":"ccb093dd-bfe8-43a6-b0be-3470585b2456","Type":"ContainerDied","Data":"724c322f479e560943ca4e76224f9820735c1745c48d7f44d525c52a1615b8aa"}
Jul 07 12:02:55.269881 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:55.269821    2570 scope.go:117] "RemoveContainer" containerID="1d1a3320f13ef33e576648aca9d08279968acf9167dfab550d64c46c58f63cff"
Jul 07 12:02:55.279577 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:55.279559    2570 scope.go:117] "RemoveContainer" containerID="1d1a3320f13ef33e576648aca9d08279968acf9167dfab550d64c46c58f63cff"
Jul 07 12:02:55.279847 ip-10-0-135-78 kubenswrapper[2570]: E0707 12:02:55.279826    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d1a3320f13ef33e576648aca9d08279968acf9167dfab550d64c46c58f63cff\": container with ID starting with 1d1a3320f13ef33e576648aca9d08279968acf9167dfab550d64c46c58f63cff not found: ID does not exist" containerID="1d1a3320f13ef33e576648aca9d08279968acf9167dfab550d64c46c58f63cff"
Jul 07 12:02:55.279913 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:55.279859    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d1a3320f13ef33e576648aca9d08279968acf9167dfab550d64c46c58f63cff"} err="failed to get container status \"1d1a3320f13ef33e576648aca9d08279968acf9167dfab550d64c46c58f63cff\": rpc error: code = NotFound desc = could not find container \"1d1a3320f13ef33e576648aca9d08279968acf9167dfab550d64c46c58f63cff\": container with ID starting with 1d1a3320f13ef33e576648aca9d08279968acf9167dfab550d64c46c58f63cff not found: ID does not exist"
Jul 07 12:02:55.285570 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:55.285524    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-02e907/test-model-02e907-kserve-bbb89ff55-4s2zl" podUID="865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1" containerName="main" containerID="cri-o://f44b874331f4e92ae4ccfb26dcf6d6f36f17c968cc5a05e03dd155e513076c3f" gracePeriod=45
Jul 07 12:02:55.292174 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:55.292149    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-302b88/test-model-302b88-kserve-7b8f66cc6c-vdh88"]
Jul 07 12:02:55.296136 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:55.296101    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-302b88/test-model-302b88-kserve-7b8f66cc6c-vdh88"]
Jul 07 12:02:55.541322 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:55.541289    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-02e907/test-model-02e907-kserve-bbb89ff55-4s2zl"
Jul 07 12:02:55.604691 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:55.604653    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1-home\") pod \"865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1\" (UID: \"865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1\") "
Jul 07 12:02:55.604864 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:55.604741    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1-dshm\") pod \"865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1\" (UID: \"865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1\") "
Jul 07 12:02:55.604864 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:55.604797    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1-tls-certs\") pod \"865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1\" (UID: \"865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1\") "
Jul 07 12:02:55.604864 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:55.604831    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1-tmp-dir\") pod \"865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1\" (UID: \"865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1\") "
Jul 07 12:02:55.605004 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:55.604884    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rwmkl\" (UniqueName: \"kubernetes.io/projected/865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1-kube-api-access-rwmkl\") pod \"865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1\" (UID: \"865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1\") "
Jul 07 12:02:55.605004 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:55.604943    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1-home" (OuterVolumeSpecName: "home") pod "865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1" (UID: "865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:02:55.605004 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:55.604976    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1-model-cache\") pod \"865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1\" (UID: \"865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1\") "
Jul 07 12:02:55.605241 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:55.605091    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1" (UID: "865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:02:55.605314 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:55.605244    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1-model-cache" (OuterVolumeSpecName: "model-cache") pod "865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1" (UID: "865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:02:55.605402 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:55.605386    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1-tmp-dir\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:02:55.605469 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:55.605402    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1-model-cache\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:02:55.605469 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:55.605412    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1-home\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:02:55.607081 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:55.607052    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1-dshm" (OuterVolumeSpecName: "dshm") pod "865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1" (UID: "865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:02:55.607200 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:55.607108    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1" (UID: "865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 12:02:55.607200 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:55.607114    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1-kube-api-access-rwmkl" (OuterVolumeSpecName: "kube-api-access-rwmkl") pod "865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1" (UID: "865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1"). InnerVolumeSpecName "kube-api-access-rwmkl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 12:02:55.706619 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:55.706587    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1-dshm\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:02:55.706619 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:55.706614    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1-tls-certs\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:02:55.706619 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:55.706625    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rwmkl\" (UniqueName: \"kubernetes.io/projected/865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1-kube-api-access-rwmkl\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:02:55.938738 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:55.938706    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ccb093dd-bfe8-43a6-b0be-3470585b2456" path="/var/lib/kubelet/pods/ccb093dd-bfe8-43a6-b0be-3470585b2456/volumes"
Jul 07 12:02:56.276154 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:56.276059    2570 generic.go:356] "Generic (PLEG): container finished" podID="865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1" containerID="f44b874331f4e92ae4ccfb26dcf6d6f36f17c968cc5a05e03dd155e513076c3f" exitCode=0
Jul 07 12:02:56.276154 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:56.276105    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-02e907/test-model-02e907-kserve-bbb89ff55-4s2zl"
Jul 07 12:02:56.276441 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:56.276148    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-02e907/test-model-02e907-kserve-bbb89ff55-4s2zl" event={"ID":"865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1","Type":"ContainerDied","Data":"f44b874331f4e92ae4ccfb26dcf6d6f36f17c968cc5a05e03dd155e513076c3f"}
Jul 07 12:02:56.276441 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:56.276195    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-02e907/test-model-02e907-kserve-bbb89ff55-4s2zl" event={"ID":"865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1","Type":"ContainerDied","Data":"791651f46ecd14f8661eae720a5a77501327b7774ef814dee5dc46e559b1a02d"}
Jul 07 12:02:56.276441 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:56.276239    2570 scope.go:117] "RemoveContainer" containerID="f44b874331f4e92ae4ccfb26dcf6d6f36f17c968cc5a05e03dd155e513076c3f"
Jul 07 12:02:56.285095 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:56.285072    2570 scope.go:117] "RemoveContainer" containerID="f44b874331f4e92ae4ccfb26dcf6d6f36f17c968cc5a05e03dd155e513076c3f"
Jul 07 12:02:56.285358 ip-10-0-135-78 kubenswrapper[2570]: E0707 12:02:56.285338    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f44b874331f4e92ae4ccfb26dcf6d6f36f17c968cc5a05e03dd155e513076c3f\": container with ID starting with f44b874331f4e92ae4ccfb26dcf6d6f36f17c968cc5a05e03dd155e513076c3f not found: ID does not exist" containerID="f44b874331f4e92ae4ccfb26dcf6d6f36f17c968cc5a05e03dd155e513076c3f"
Jul 07 12:02:56.285428 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:56.285368    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f44b874331f4e92ae4ccfb26dcf6d6f36f17c968cc5a05e03dd155e513076c3f"} err="failed to get container status \"f44b874331f4e92ae4ccfb26dcf6d6f36f17c968cc5a05e03dd155e513076c3f\": rpc error: code = NotFound desc = could not find container \"f44b874331f4e92ae4ccfb26dcf6d6f36f17c968cc5a05e03dd155e513076c3f\": container with ID starting with f44b874331f4e92ae4ccfb26dcf6d6f36f17c968cc5a05e03dd155e513076c3f not found: ID does not exist"
Jul 07 12:02:56.295293 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:56.295262    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-02e907/test-model-02e907-kserve-bbb89ff55-4s2zl"]
Jul 07 12:02:56.298913 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:56.298891    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-02e907/test-model-02e907-kserve-bbb89ff55-4s2zl"]
Jul 07 12:02:57.938774 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:02:57.938735    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1" path="/var/lib/kubelet/pods/865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1/volumes"
Jul 07 12:03:29.939087 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:29.939056    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt"]
Jul 07 12:03:29.939608 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:29.939304    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt" podUID="72cd0119-5c4d-459a-938b-a900543eee7d" containerName="istio-proxy" containerID="cri-o://30ea07dac0c7ffef9e6fa840e413a7a11a06b5375cb0792f7e3cc247ffe312b0" gracePeriod=30
Jul 07 12:03:30.609185 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:30.609147    2570 patch_prober.go:28] interesting pod/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 07 12:03:30.609382 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:30.609239    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt" podUID="72cd0119-5c4d-459a-938b-a900543eee7d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 12:03:35.205434 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:35.205358    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt"
Jul 07 12:03:35.292045 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:35.292004    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72cd0119-5c4d-459a-938b-a900543eee7d-credential-socket\") pod \"72cd0119-5c4d-459a-938b-a900543eee7d\" (UID: \"72cd0119-5c4d-459a-938b-a900543eee7d\") "
Jul 07 12:03:35.292266 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:35.292082    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72cd0119-5c4d-459a-938b-a900543eee7d-workload-certs\") pod \"72cd0119-5c4d-459a-938b-a900543eee7d\" (UID: \"72cd0119-5c4d-459a-938b-a900543eee7d\") "
Jul 07 12:03:35.292266 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:35.292126    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xtmcl\" (UniqueName: \"kubernetes.io/projected/72cd0119-5c4d-459a-938b-a900543eee7d-kube-api-access-xtmcl\") pod \"72cd0119-5c4d-459a-938b-a900543eee7d\" (UID: \"72cd0119-5c4d-459a-938b-a900543eee7d\") "
Jul 07 12:03:35.292266 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:35.292159    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72cd0119-5c4d-459a-938b-a900543eee7d-istio-data\") pod \"72cd0119-5c4d-459a-938b-a900543eee7d\" (UID: \"72cd0119-5c4d-459a-938b-a900543eee7d\") "
Jul 07 12:03:35.292266 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:35.292195    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72cd0119-5c4d-459a-938b-a900543eee7d-istio-podinfo\") pod \"72cd0119-5c4d-459a-938b-a900543eee7d\" (UID: \"72cd0119-5c4d-459a-938b-a900543eee7d\") "
Jul 07 12:03:35.292266 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:35.292259    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72cd0119-5c4d-459a-938b-a900543eee7d-istio-envoy\") pod \"72cd0119-5c4d-459a-938b-a900543eee7d\" (UID: \"72cd0119-5c4d-459a-938b-a900543eee7d\") "
Jul 07 12:03:35.292528 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:35.292344    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72cd0119-5c4d-459a-938b-a900543eee7d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "72cd0119-5c4d-459a-938b-a900543eee7d" (UID: "72cd0119-5c4d-459a-938b-a900543eee7d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:03:35.292528 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:35.292416    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72cd0119-5c4d-459a-938b-a900543eee7d-istiod-ca-cert\") pod \"72cd0119-5c4d-459a-938b-a900543eee7d\" (UID: \"72cd0119-5c4d-459a-938b-a900543eee7d\") "
Jul 07 12:03:35.292528 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:35.292474    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72cd0119-5c4d-459a-938b-a900543eee7d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "72cd0119-5c4d-459a-938b-a900543eee7d" (UID: "72cd0119-5c4d-459a-938b-a900543eee7d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:03:35.292696 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:35.292560    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72cd0119-5c4d-459a-938b-a900543eee7d-istio-token\") pod \"72cd0119-5c4d-459a-938b-a900543eee7d\" (UID: \"72cd0119-5c4d-459a-938b-a900543eee7d\") "
Jul 07 12:03:35.292696 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:35.292653    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72cd0119-5c4d-459a-938b-a900543eee7d-workload-socket\") pod \"72cd0119-5c4d-459a-938b-a900543eee7d\" (UID: \"72cd0119-5c4d-459a-938b-a900543eee7d\") "
Jul 07 12:03:35.292851 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:35.292825    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72cd0119-5c4d-459a-938b-a900543eee7d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "72cd0119-5c4d-459a-938b-a900543eee7d" (UID: "72cd0119-5c4d-459a-938b-a900543eee7d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 12:03:35.293065 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:35.293046    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72cd0119-5c4d-459a-938b-a900543eee7d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "72cd0119-5c4d-459a-938b-a900543eee7d" (UID: "72cd0119-5c4d-459a-938b-a900543eee7d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:03:35.293142 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:35.293126    2570 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72cd0119-5c4d-459a-938b-a900543eee7d-credential-socket\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:03:35.293191 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:35.293147    2570 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72cd0119-5c4d-459a-938b-a900543eee7d-workload-certs\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:03:35.293191 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:35.293143    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72cd0119-5c4d-459a-938b-a900543eee7d-istio-data" (OuterVolumeSpecName: "istio-data") pod "72cd0119-5c4d-459a-938b-a900543eee7d" (UID: "72cd0119-5c4d-459a-938b-a900543eee7d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:03:35.293191 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:35.293162    2570 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72cd0119-5c4d-459a-938b-a900543eee7d-istiod-ca-cert\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:03:35.294597 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:35.294573    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72cd0119-5c4d-459a-938b-a900543eee7d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "72cd0119-5c4d-459a-938b-a900543eee7d" (UID: "72cd0119-5c4d-459a-938b-a900543eee7d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:03:35.294692 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:35.294603    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72cd0119-5c4d-459a-938b-a900543eee7d-kube-api-access-xtmcl" (OuterVolumeSpecName: "kube-api-access-xtmcl") pod "72cd0119-5c4d-459a-938b-a900543eee7d" (UID: "72cd0119-5c4d-459a-938b-a900543eee7d"). InnerVolumeSpecName "kube-api-access-xtmcl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 12:03:35.294808 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:35.294785    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/72cd0119-5c4d-459a-938b-a900543eee7d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "72cd0119-5c4d-459a-938b-a900543eee7d" (UID: "72cd0119-5c4d-459a-938b-a900543eee7d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 12:03:35.294876 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:35.294843    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72cd0119-5c4d-459a-938b-a900543eee7d-istio-token" (OuterVolumeSpecName: "istio-token") pod "72cd0119-5c4d-459a-938b-a900543eee7d" (UID: "72cd0119-5c4d-459a-938b-a900543eee7d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 12:03:35.394644 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:35.394611    2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72cd0119-5c4d-459a-938b-a900543eee7d-istio-token\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:03:35.394644 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:35.394639    2570 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72cd0119-5c4d-459a-938b-a900543eee7d-workload-socket\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:03:35.394644 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:35.394651    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xtmcl\" (UniqueName: \"kubernetes.io/projected/72cd0119-5c4d-459a-938b-a900543eee7d-kube-api-access-xtmcl\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:03:35.394885 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:35.394661    2570 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72cd0119-5c4d-459a-938b-a900543eee7d-istio-data\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:03:35.394885 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:35.394669    2570 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72cd0119-5c4d-459a-938b-a900543eee7d-istio-podinfo\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:03:35.394885 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:35.394677    2570 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72cd0119-5c4d-459a-938b-a900543eee7d-istio-envoy\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:03:35.501737 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:35.501620    2570 generic.go:356] "Generic (PLEG): container finished" podID="72cd0119-5c4d-459a-938b-a900543eee7d" containerID="30ea07dac0c7ffef9e6fa840e413a7a11a06b5375cb0792f7e3cc247ffe312b0" exitCode=0
Jul 07 12:03:35.501737 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:35.501720    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt" event={"ID":"72cd0119-5c4d-459a-938b-a900543eee7d","Type":"ContainerDied","Data":"30ea07dac0c7ffef9e6fa840e413a7a11a06b5375cb0792f7e3cc247ffe312b0"}
Jul 07 12:03:35.501994 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:35.501806    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt" event={"ID":"72cd0119-5c4d-459a-938b-a900543eee7d","Type":"ContainerDied","Data":"e043b2754efeef616638aa7eab784272982c34dfa739002793daa94f0fe4b010"}
Jul 07 12:03:35.501994 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:35.501835    2570 scope.go:117] "RemoveContainer" containerID="30ea07dac0c7ffef9e6fa840e413a7a11a06b5375cb0792f7e3cc247ffe312b0"
Jul 07 12:03:35.502115 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:35.502021    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt"
Jul 07 12:03:35.513352 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:35.513332    2570 scope.go:117] "RemoveContainer" containerID="f9a5e5b6098d67e334d03cd7e8b5aebefb4160a0d49ca54460cf1c598093cec5"
Jul 07 12:03:35.522574 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:35.522444    2570 scope.go:117] "RemoveContainer" containerID="30ea07dac0c7ffef9e6fa840e413a7a11a06b5375cb0792f7e3cc247ffe312b0"
Jul 07 12:03:35.524563 ip-10-0-135-78 kubenswrapper[2570]: E0707 12:03:35.524527    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30ea07dac0c7ffef9e6fa840e413a7a11a06b5375cb0792f7e3cc247ffe312b0\": container with ID starting with 30ea07dac0c7ffef9e6fa840e413a7a11a06b5375cb0792f7e3cc247ffe312b0 not found: ID does not exist" containerID="30ea07dac0c7ffef9e6fa840e413a7a11a06b5375cb0792f7e3cc247ffe312b0"
Jul 07 12:03:35.524686 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:35.524575    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30ea07dac0c7ffef9e6fa840e413a7a11a06b5375cb0792f7e3cc247ffe312b0"} err="failed to get container status \"30ea07dac0c7ffef9e6fa840e413a7a11a06b5375cb0792f7e3cc247ffe312b0\": rpc error: code = NotFound desc = could not find container \"30ea07dac0c7ffef9e6fa840e413a7a11a06b5375cb0792f7e3cc247ffe312b0\": container with ID starting with 30ea07dac0c7ffef9e6fa840e413a7a11a06b5375cb0792f7e3cc247ffe312b0 not found: ID does not exist"
Jul 07 12:03:35.524686 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:35.524598    2570 scope.go:117] "RemoveContainer" containerID="f9a5e5b6098d67e334d03cd7e8b5aebefb4160a0d49ca54460cf1c598093cec5"
Jul 07 12:03:35.525153 ip-10-0-135-78 kubenswrapper[2570]: E0707 12:03:35.525122    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9a5e5b6098d67e334d03cd7e8b5aebefb4160a0d49ca54460cf1c598093cec5\": container with ID starting with f9a5e5b6098d67e334d03cd7e8b5aebefb4160a0d49ca54460cf1c598093cec5 not found: ID does not exist" containerID="f9a5e5b6098d67e334d03cd7e8b5aebefb4160a0d49ca54460cf1c598093cec5"
Jul 07 12:03:35.525302 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:35.525158    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9a5e5b6098d67e334d03cd7e8b5aebefb4160a0d49ca54460cf1c598093cec5"} err="failed to get container status \"f9a5e5b6098d67e334d03cd7e8b5aebefb4160a0d49ca54460cf1c598093cec5\": rpc error: code = NotFound desc = could not find container \"f9a5e5b6098d67e334d03cd7e8b5aebefb4160a0d49ca54460cf1c598093cec5\": container with ID starting with f9a5e5b6098d67e334d03cd7e8b5aebefb4160a0d49ca54460cf1c598093cec5 not found: ID does not exist"
Jul 07 12:03:35.527431 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:35.527405    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt"]
Jul 07 12:03:35.529525 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:35.529505    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-02e907-openshift-default-54b49cf577-x2rkt"]
Jul 07 12:03:35.938181 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:35.938149    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72cd0119-5c4d-459a-938b-a900543eee7d" path="/var/lib/kubelet/pods/72cd0119-5c4d-459a-938b-a900543eee7d/volumes"
Jul 07 12:03:39.091508 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:39.091476    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797"]
Jul 07 12:03:39.092005 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:39.091986    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ccb093dd-bfe8-43a6-b0be-3470585b2456" containerName="main"
Jul 07 12:03:39.092079 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:39.092006    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="ccb093dd-bfe8-43a6-b0be-3470585b2456" containerName="main"
Jul 07 12:03:39.092079 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:39.092031    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1" containerName="main"
Jul 07 12:03:39.092079 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:39.092041    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1" containerName="main"
Jul 07 12:03:39.092079 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:39.092056    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72cd0119-5c4d-459a-938b-a900543eee7d" containerName="istio-proxy"
Jul 07 12:03:39.092079 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:39.092065    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="72cd0119-5c4d-459a-938b-a900543eee7d" containerName="istio-proxy"
Jul 07 12:03:39.092358 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:39.092087    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72cd0119-5c4d-459a-938b-a900543eee7d" containerName="istio-proxy"
Jul 07 12:03:39.092358 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:39.092096    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="72cd0119-5c4d-459a-938b-a900543eee7d" containerName="istio-proxy"
Jul 07 12:03:39.092358 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:39.092191    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="72cd0119-5c4d-459a-938b-a900543eee7d" containerName="istio-proxy"
Jul 07 12:03:39.092358 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:39.092204    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="865b0cdd-0ab5-45da-bb32-0fc9e6f7b2d1" containerName="main"
Jul 07 12:03:39.092358 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:39.092233    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="ccb093dd-bfe8-43a6-b0be-3470585b2456" containerName="main"
Jul 07 12:03:39.097009 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:39.096986    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797"
Jul 07 12:03:39.099608 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:39.099586    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-a1dd3f-openshift-default-dockercfg-bf4hd\""
Jul 07 12:03:39.109276 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:39.109246    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797"]
Jul 07 12:03:39.231969 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:39.231925    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5bccca9d-1571-45c6-8164-45f0c6648bd9-istiod-ca-cert\") pod \"e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797\" (UID: \"5bccca9d-1571-45c6-8164-45f0c6648bd9\") " pod="openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797"
Jul 07 12:03:39.232192 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:39.231983    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5bccca9d-1571-45c6-8164-45f0c6648bd9-istio-token\") pod \"e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797\" (UID: \"5bccca9d-1571-45c6-8164-45f0c6648bd9\") " pod="openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797"
Jul 07 12:03:39.232192 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:39.232044    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bccca9d-1571-45c6-8164-45f0c6648bd9-workload-socket\") pod \"e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797\" (UID: \"5bccca9d-1571-45c6-8164-45f0c6648bd9\") " pod="openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797"
Jul 07 12:03:39.232192 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:39.232078    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5bccca9d-1571-45c6-8164-45f0c6648bd9-istio-envoy\") pod \"e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797\" (UID: \"5bccca9d-1571-45c6-8164-45f0c6648bd9\") " pod="openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797"
Jul 07 12:03:39.232192 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:39.232115    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5bccca9d-1571-45c6-8164-45f0c6648bd9-istio-data\") pod \"e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797\" (UID: \"5bccca9d-1571-45c6-8164-45f0c6648bd9\") " pod="openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797"
Jul 07 12:03:39.232192 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:39.232151    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bccca9d-1571-45c6-8164-45f0c6648bd9-credential-socket\") pod \"e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797\" (UID: \"5bccca9d-1571-45c6-8164-45f0c6648bd9\") " pod="openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797"
Jul 07 12:03:39.232436 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:39.232258    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5bccca9d-1571-45c6-8164-45f0c6648bd9-workload-certs\") pod \"e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797\" (UID: \"5bccca9d-1571-45c6-8164-45f0c6648bd9\") " pod="openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797"
Jul 07 12:03:39.232436 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:39.232299    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5bccca9d-1571-45c6-8164-45f0c6648bd9-istio-podinfo\") pod \"e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797\" (UID: \"5bccca9d-1571-45c6-8164-45f0c6648bd9\") " pod="openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797"
Jul 07 12:03:39.232436 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:39.232366    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tj4ww\" (UniqueName: \"kubernetes.io/projected/5bccca9d-1571-45c6-8164-45f0c6648bd9-kube-api-access-tj4ww\") pod \"e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797\" (UID: \"5bccca9d-1571-45c6-8164-45f0c6648bd9\") " pod="openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797"
Jul 07 12:03:39.333317 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:39.333281    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bccca9d-1571-45c6-8164-45f0c6648bd9-workload-socket\") pod \"e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797\" (UID: \"5bccca9d-1571-45c6-8164-45f0c6648bd9\") " pod="openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797"
Jul 07 12:03:39.333531 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:39.333331    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5bccca9d-1571-45c6-8164-45f0c6648bd9-istio-envoy\") pod \"e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797\" (UID: \"5bccca9d-1571-45c6-8164-45f0c6648bd9\") " pod="openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797"
Jul 07 12:03:39.333531 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:39.333450    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5bccca9d-1571-45c6-8164-45f0c6648bd9-istio-data\") pod \"e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797\" (UID: \"5bccca9d-1571-45c6-8164-45f0c6648bd9\") " pod="openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797"
Jul 07 12:03:39.333531 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:39.333515    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bccca9d-1571-45c6-8164-45f0c6648bd9-credential-socket\") pod \"e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797\" (UID: \"5bccca9d-1571-45c6-8164-45f0c6648bd9\") " pod="openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797"
Jul 07 12:03:39.333776 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:39.333543    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5bccca9d-1571-45c6-8164-45f0c6648bd9-workload-certs\") pod \"e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797\" (UID: \"5bccca9d-1571-45c6-8164-45f0c6648bd9\") " pod="openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797"
Jul 07 12:03:39.333776 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:39.333565    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5bccca9d-1571-45c6-8164-45f0c6648bd9-istio-podinfo\") pod \"e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797\" (UID: \"5bccca9d-1571-45c6-8164-45f0c6648bd9\") " pod="openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797"
Jul 07 12:03:39.333776 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:39.333610    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tj4ww\" (UniqueName: \"kubernetes.io/projected/5bccca9d-1571-45c6-8164-45f0c6648bd9-kube-api-access-tj4ww\") pod \"e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797\" (UID: \"5bccca9d-1571-45c6-8164-45f0c6648bd9\") " pod="openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797"
Jul 07 12:03:39.333776 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:39.333646    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5bccca9d-1571-45c6-8164-45f0c6648bd9-istiod-ca-cert\") pod \"e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797\" (UID: \"5bccca9d-1571-45c6-8164-45f0c6648bd9\") " pod="openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797"
Jul 07 12:03:39.333988 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:39.333797    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bccca9d-1571-45c6-8164-45f0c6648bd9-workload-socket\") pod \"e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797\" (UID: \"5bccca9d-1571-45c6-8164-45f0c6648bd9\") " pod="openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797"
Jul 07 12:03:39.333988 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:39.333854    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5bccca9d-1571-45c6-8164-45f0c6648bd9-istio-data\") pod \"e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797\" (UID: \"5bccca9d-1571-45c6-8164-45f0c6648bd9\") " pod="openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797"
Jul 07 12:03:39.333988 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:39.333920    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bccca9d-1571-45c6-8164-45f0c6648bd9-credential-socket\") pod \"e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797\" (UID: \"5bccca9d-1571-45c6-8164-45f0c6648bd9\") " pod="openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797"
Jul 07 12:03:39.333988 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:39.333934    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5bccca9d-1571-45c6-8164-45f0c6648bd9-workload-certs\") pod \"e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797\" (UID: \"5bccca9d-1571-45c6-8164-45f0c6648bd9\") " pod="openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797"
Jul 07 12:03:39.334207 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:39.334037    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5bccca9d-1571-45c6-8164-45f0c6648bd9-istio-token\") pod \"e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797\" (UID: \"5bccca9d-1571-45c6-8164-45f0c6648bd9\") " pod="openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797"
Jul 07 12:03:39.334539 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:39.334517    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5bccca9d-1571-45c6-8164-45f0c6648bd9-istiod-ca-cert\") pod \"e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797\" (UID: \"5bccca9d-1571-45c6-8164-45f0c6648bd9\") " pod="openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797"
Jul 07 12:03:39.335759 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:39.335728    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5bccca9d-1571-45c6-8164-45f0c6648bd9-istio-envoy\") pod \"e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797\" (UID: \"5bccca9d-1571-45c6-8164-45f0c6648bd9\") " pod="openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797"
Jul 07 12:03:39.336046 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:39.336026    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5bccca9d-1571-45c6-8164-45f0c6648bd9-istio-podinfo\") pod \"e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797\" (UID: \"5bccca9d-1571-45c6-8164-45f0c6648bd9\") " pod="openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797"
Jul 07 12:03:39.341831 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:39.341779    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tj4ww\" (UniqueName: \"kubernetes.io/projected/5bccca9d-1571-45c6-8164-45f0c6648bd9-kube-api-access-tj4ww\") pod \"e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797\" (UID: \"5bccca9d-1571-45c6-8164-45f0c6648bd9\") " pod="openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797"
Jul 07 12:03:39.341905 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:39.341872    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5bccca9d-1571-45c6-8164-45f0c6648bd9-istio-token\") pod \"e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797\" (UID: \"5bccca9d-1571-45c6-8164-45f0c6648bd9\") " pod="openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797"
Jul 07 12:03:39.408442 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:39.408403    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797"
Jul 07 12:03:39.551850 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:39.551821    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797"]
Jul 07 12:03:39.553618 ip-10-0-135-78 kubenswrapper[2570]: W0707 12:03:39.553581    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bccca9d_1571_45c6_8164_45f0c6648bd9.slice/crio-9060cf41f67285d572480ac150c9b20a37f213c085a43c984cc02844c31183af WatchSource:0}: Error finding container 9060cf41f67285d572480ac150c9b20a37f213c085a43c984cc02844c31183af: Status 404 returned error can't find the container with id 9060cf41f67285d572480ac150c9b20a37f213c085a43c984cc02844c31183af
Jul 07 12:03:39.555976 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:39.555937    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 12:03:39.556062 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:39.556031    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 12:03:39.556099 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:39.556075    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 12:03:40.531655 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:40.531614    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797" event={"ID":"5bccca9d-1571-45c6-8164-45f0c6648bd9","Type":"ContainerStarted","Data":"b91978cf66437baf4ce3f825899ce41f9b3ca8fa16d744c7e3f6f9b296d082cb"}
Jul 07 12:03:40.532072 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:40.531672    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797" event={"ID":"5bccca9d-1571-45c6-8164-45f0c6648bd9","Type":"ContainerStarted","Data":"9060cf41f67285d572480ac150c9b20a37f213c085a43c984cc02844c31183af"}
Jul 07 12:03:40.557918 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:40.557864    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797" podStartSLOduration=1.557845812 podStartE2EDuration="1.557845812s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:03:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:03:40.55533322 +0000 UTC m=+2208.528973650" watchObservedRunningTime="2026-07-07 12:03:40.557845812 +0000 UTC m=+2208.531486221"
Jul 07 12:03:41.409026 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:41.408991    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797"
Jul 07 12:03:41.413959 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:41.413930    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797"
Jul 07 12:03:41.537454 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:41.537425    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797"
Jul 07 12:03:41.538363 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:41.538338    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797"
Jul 07 12:03:57.579413 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:57.579368    2570 patch_prober.go:28] interesting pod/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 07 12:03:57.579849 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:57.579446    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797" podUID="5bccca9d-1571-45c6-8164-45f0c6648bd9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 07 12:03:58.627272 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:58.627236    2570 generic.go:356] "Generic (PLEG): container finished" podID="5bccca9d-1571-45c6-8164-45f0c6648bd9" containerID="b91978cf66437baf4ce3f825899ce41f9b3ca8fa16d744c7e3f6f9b296d082cb" exitCode=0
Jul 07 12:03:58.627702 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:58.627310    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797" event={"ID":"5bccca9d-1571-45c6-8164-45f0c6648bd9","Type":"ContainerDied","Data":"b91978cf66437baf4ce3f825899ce41f9b3ca8fa16d744c7e3f6f9b296d082cb"}
Jul 07 12:03:58.627746 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:58.627727    2570 scope.go:117] "RemoveContainer" containerID="b91978cf66437baf4ce3f825899ce41f9b3ca8fa16d744c7e3f6f9b296d082cb"
Jul 07 12:03:58.628885 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:58.628858    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 12:03:58.628958 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:58.628939    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 12:03:58.629000 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:58.628988    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 12:03:59.640355 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:03:59.640308    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797" event={"ID":"5bccca9d-1571-45c6-8164-45f0c6648bd9","Type":"ContainerStarted","Data":"ee2bad93f5b33e65611c0c5d231f5cd0441e06781cd3cfccf7899d5908fd5196"}
Jul 07 12:04:00.409322 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:00.409280    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797"
Jul 07 12:04:00.409562 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:00.409542    2570 patch_prober.go:28] interesting pod/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Jul 07 12:04:00.409623 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:00.409603    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797" podUID="5bccca9d-1571-45c6-8164-45f0c6648bd9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Jul 07 12:04:01.409848 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:01.409808    2570 patch_prober.go:28] interesting pod/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Jul 07 12:04:01.410320 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:01.409874    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797" podUID="5bccca9d-1571-45c6-8164-45f0c6648bd9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Jul 07 12:04:02.408977 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:02.408939    2570 patch_prober.go:28] interesting pod/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Jul 07 12:04:02.409198 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:02.409000    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797" podUID="5bccca9d-1571-45c6-8164-45f0c6648bd9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Jul 07 12:04:03.408884 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:03.408842    2570 patch_prober.go:28] interesting pod/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Jul 07 12:04:03.409331 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:03.408909    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797" podUID="5bccca9d-1571-45c6-8164-45f0c6648bd9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Jul 07 12:04:04.412961 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:04.412929    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797"
Jul 07 12:04:04.413430 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:04.413204    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797"
Jul 07 12:04:04.413980 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:04.413963    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797"
Jul 07 12:04:08.445037 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:08.445001    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797"]
Jul 07 12:04:08.445541 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:08.445232    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797" podUID="5bccca9d-1571-45c6-8164-45f0c6648bd9" containerName="istio-proxy" containerID="cri-o://ee2bad93f5b33e65611c0c5d231f5cd0441e06781cd3cfccf7899d5908fd5196" gracePeriod=30
Jul 07 12:04:13.695386 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:13.695358    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797"
Jul 07 12:04:13.717760 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:13.717674    2570 generic.go:356] "Generic (PLEG): container finished" podID="5bccca9d-1571-45c6-8164-45f0c6648bd9" containerID="ee2bad93f5b33e65611c0c5d231f5cd0441e06781cd3cfccf7899d5908fd5196" exitCode=0
Jul 07 12:04:13.717760 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:13.717728    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797"
Jul 07 12:04:13.717993 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:13.717759    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797" event={"ID":"5bccca9d-1571-45c6-8164-45f0c6648bd9","Type":"ContainerDied","Data":"ee2bad93f5b33e65611c0c5d231f5cd0441e06781cd3cfccf7899d5908fd5196"}
Jul 07 12:04:13.717993 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:13.717820    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797" event={"ID":"5bccca9d-1571-45c6-8164-45f0c6648bd9","Type":"ContainerDied","Data":"9060cf41f67285d572480ac150c9b20a37f213c085a43c984cc02844c31183af"}
Jul 07 12:04:13.717993 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:13.717851    2570 scope.go:117] "RemoveContainer" containerID="ee2bad93f5b33e65611c0c5d231f5cd0441e06781cd3cfccf7899d5908fd5196"
Jul 07 12:04:13.728178 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:13.728158    2570 scope.go:117] "RemoveContainer" containerID="b91978cf66437baf4ce3f825899ce41f9b3ca8fa16d744c7e3f6f9b296d082cb"
Jul 07 12:04:13.738760 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:13.738736    2570 scope.go:117] "RemoveContainer" containerID="ee2bad93f5b33e65611c0c5d231f5cd0441e06781cd3cfccf7899d5908fd5196"
Jul 07 12:04:13.739081 ip-10-0-135-78 kubenswrapper[2570]: E0707 12:04:13.739057    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee2bad93f5b33e65611c0c5d231f5cd0441e06781cd3cfccf7899d5908fd5196\": container with ID starting with ee2bad93f5b33e65611c0c5d231f5cd0441e06781cd3cfccf7899d5908fd5196 not found: ID does not exist" containerID="ee2bad93f5b33e65611c0c5d231f5cd0441e06781cd3cfccf7899d5908fd5196"
Jul 07 12:04:13.739189 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:13.739097    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee2bad93f5b33e65611c0c5d231f5cd0441e06781cd3cfccf7899d5908fd5196"} err="failed to get container status \"ee2bad93f5b33e65611c0c5d231f5cd0441e06781cd3cfccf7899d5908fd5196\": rpc error: code = NotFound desc = could not find container \"ee2bad93f5b33e65611c0c5d231f5cd0441e06781cd3cfccf7899d5908fd5196\": container with ID starting with ee2bad93f5b33e65611c0c5d231f5cd0441e06781cd3cfccf7899d5908fd5196 not found: ID does not exist"
Jul 07 12:04:13.739189 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:13.739116    2570 scope.go:117] "RemoveContainer" containerID="b91978cf66437baf4ce3f825899ce41f9b3ca8fa16d744c7e3f6f9b296d082cb"
Jul 07 12:04:13.739390 ip-10-0-135-78 kubenswrapper[2570]: E0707 12:04:13.739371    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b91978cf66437baf4ce3f825899ce41f9b3ca8fa16d744c7e3f6f9b296d082cb\": container with ID starting with b91978cf66437baf4ce3f825899ce41f9b3ca8fa16d744c7e3f6f9b296d082cb not found: ID does not exist" containerID="b91978cf66437baf4ce3f825899ce41f9b3ca8fa16d744c7e3f6f9b296d082cb"
Jul 07 12:04:13.739453 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:13.739394    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b91978cf66437baf4ce3f825899ce41f9b3ca8fa16d744c7e3f6f9b296d082cb"} err="failed to get container status \"b91978cf66437baf4ce3f825899ce41f9b3ca8fa16d744c7e3f6f9b296d082cb\": rpc error: code = NotFound desc = could not find container \"b91978cf66437baf4ce3f825899ce41f9b3ca8fa16d744c7e3f6f9b296d082cb\": container with ID starting with b91978cf66437baf4ce3f825899ce41f9b3ca8fa16d744c7e3f6f9b296d082cb not found: ID does not exist"
Jul 07 12:04:13.772670 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:13.772632    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5bccca9d-1571-45c6-8164-45f0c6648bd9-istio-podinfo\") pod \"5bccca9d-1571-45c6-8164-45f0c6648bd9\" (UID: \"5bccca9d-1571-45c6-8164-45f0c6648bd9\") "
Jul 07 12:04:13.772670 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:13.772675    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tj4ww\" (UniqueName: \"kubernetes.io/projected/5bccca9d-1571-45c6-8164-45f0c6648bd9-kube-api-access-tj4ww\") pod \"5bccca9d-1571-45c6-8164-45f0c6648bd9\" (UID: \"5bccca9d-1571-45c6-8164-45f0c6648bd9\") "
Jul 07 12:04:13.772912 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:13.772698    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5bccca9d-1571-45c6-8164-45f0c6648bd9-workload-certs\") pod \"5bccca9d-1571-45c6-8164-45f0c6648bd9\" (UID: \"5bccca9d-1571-45c6-8164-45f0c6648bd9\") "
Jul 07 12:04:13.772912 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:13.772714    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5bccca9d-1571-45c6-8164-45f0c6648bd9-istio-token\") pod \"5bccca9d-1571-45c6-8164-45f0c6648bd9\" (UID: \"5bccca9d-1571-45c6-8164-45f0c6648bd9\") "
Jul 07 12:04:13.772912 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:13.772780    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5bccca9d-1571-45c6-8164-45f0c6648bd9-istiod-ca-cert\") pod \"5bccca9d-1571-45c6-8164-45f0c6648bd9\" (UID: \"5bccca9d-1571-45c6-8164-45f0c6648bd9\") "
Jul 07 12:04:13.772912 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:13.772813    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bccca9d-1571-45c6-8164-45f0c6648bd9-credential-socket\") pod \"5bccca9d-1571-45c6-8164-45f0c6648bd9\" (UID: \"5bccca9d-1571-45c6-8164-45f0c6648bd9\") "
Jul 07 12:04:13.772912 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:13.772842    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bccca9d-1571-45c6-8164-45f0c6648bd9-workload-socket\") pod \"5bccca9d-1571-45c6-8164-45f0c6648bd9\" (UID: \"5bccca9d-1571-45c6-8164-45f0c6648bd9\") "
Jul 07 12:04:13.772912 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:13.772865    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5bccca9d-1571-45c6-8164-45f0c6648bd9-istio-envoy\") pod \"5bccca9d-1571-45c6-8164-45f0c6648bd9\" (UID: \"5bccca9d-1571-45c6-8164-45f0c6648bd9\") "
Jul 07 12:04:13.773261 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:13.772947    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5bccca9d-1571-45c6-8164-45f0c6648bd9-istio-data\") pod \"5bccca9d-1571-45c6-8164-45f0c6648bd9\" (UID: \"5bccca9d-1571-45c6-8164-45f0c6648bd9\") "
Jul 07 12:04:13.773261 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:13.773039    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bccca9d-1571-45c6-8164-45f0c6648bd9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5bccca9d-1571-45c6-8164-45f0c6648bd9" (UID: "5bccca9d-1571-45c6-8164-45f0c6648bd9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:04:13.773261 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:13.773130    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bccca9d-1571-45c6-8164-45f0c6648bd9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5bccca9d-1571-45c6-8164-45f0c6648bd9" (UID: "5bccca9d-1571-45c6-8164-45f0c6648bd9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:04:13.773261 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:13.773147    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bccca9d-1571-45c6-8164-45f0c6648bd9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5bccca9d-1571-45c6-8164-45f0c6648bd9" (UID: "5bccca9d-1571-45c6-8164-45f0c6648bd9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:04:13.773261 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:13.773191    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5bccca9d-1571-45c6-8164-45f0c6648bd9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5bccca9d-1571-45c6-8164-45f0c6648bd9" (UID: "5bccca9d-1571-45c6-8164-45f0c6648bd9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 12:04:13.773593 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:13.773570    2570 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5bccca9d-1571-45c6-8164-45f0c6648bd9-istiod-ca-cert\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:04:13.773677 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:13.773595    2570 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bccca9d-1571-45c6-8164-45f0c6648bd9-credential-socket\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:04:13.773677 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:13.773610    2570 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bccca9d-1571-45c6-8164-45f0c6648bd9-workload-socket\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:04:13.773677 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:13.773624    2570 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5bccca9d-1571-45c6-8164-45f0c6648bd9-workload-certs\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:04:13.774044 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:13.774016    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bccca9d-1571-45c6-8164-45f0c6648bd9-istio-data" (OuterVolumeSpecName: "istio-data") pod "5bccca9d-1571-45c6-8164-45f0c6648bd9" (UID: "5bccca9d-1571-45c6-8164-45f0c6648bd9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:04:13.775058 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:13.775034    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5bccca9d-1571-45c6-8164-45f0c6648bd9-istio-token" (OuterVolumeSpecName: "istio-token") pod "5bccca9d-1571-45c6-8164-45f0c6648bd9" (UID: "5bccca9d-1571-45c6-8164-45f0c6648bd9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 12:04:13.775156 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:13.775064    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5bccca9d-1571-45c6-8164-45f0c6648bd9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5bccca9d-1571-45c6-8164-45f0c6648bd9" (UID: "5bccca9d-1571-45c6-8164-45f0c6648bd9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 12:04:13.775246 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:13.775200    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5bccca9d-1571-45c6-8164-45f0c6648bd9-kube-api-access-tj4ww" (OuterVolumeSpecName: "kube-api-access-tj4ww") pod "5bccca9d-1571-45c6-8164-45f0c6648bd9" (UID: "5bccca9d-1571-45c6-8164-45f0c6648bd9"). InnerVolumeSpecName "kube-api-access-tj4ww". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 12:04:13.775294 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:13.775205    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5bccca9d-1571-45c6-8164-45f0c6648bd9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5bccca9d-1571-45c6-8164-45f0c6648bd9" (UID: "5bccca9d-1571-45c6-8164-45f0c6648bd9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:04:13.874801 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:13.874758    2570 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5bccca9d-1571-45c6-8164-45f0c6648bd9-istio-podinfo\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:04:13.874801 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:13.874795    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tj4ww\" (UniqueName: \"kubernetes.io/projected/5bccca9d-1571-45c6-8164-45f0c6648bd9-kube-api-access-tj4ww\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:04:13.874801 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:13.874809    2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5bccca9d-1571-45c6-8164-45f0c6648bd9-istio-token\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:04:13.875052 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:13.874823    2570 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5bccca9d-1571-45c6-8164-45f0c6648bd9-istio-envoy\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:04:13.875052 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:13.874834    2570 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5bccca9d-1571-45c6-8164-45f0c6648bd9-istio-data\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:04:14.036003 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:14.035886    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797"]
Jul 07 12:04:14.039250 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:14.039222    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-a1dd3f-openshift-default-8fc584cc9-kl797"]
Jul 07 12:04:15.938491 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:15.938453    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5bccca9d-1571-45c6-8164-45f0c6648bd9" path="/var/lib/kubelet/pods/5bccca9d-1571-45c6-8164-45f0c6648bd9/volumes"
Jul 07 12:04:17.126604 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:17.126567    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp"]
Jul 07 12:04:17.126982 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:17.126931    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bccca9d-1571-45c6-8164-45f0c6648bd9" containerName="istio-proxy"
Jul 07 12:04:17.126982 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:17.126944    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bccca9d-1571-45c6-8164-45f0c6648bd9" containerName="istio-proxy"
Jul 07 12:04:17.126982 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:17.126963    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5bccca9d-1571-45c6-8164-45f0c6648bd9" containerName="istio-proxy"
Jul 07 12:04:17.126982 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:17.126968    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="5bccca9d-1571-45c6-8164-45f0c6648bd9" containerName="istio-proxy"
Jul 07 12:04:17.127115 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:17.127023    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="5bccca9d-1571-45c6-8164-45f0c6648bd9" containerName="istio-proxy"
Jul 07 12:04:17.127115 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:17.127031    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="72cd0119-5c4d-459a-938b-a900543eee7d" containerName="istio-proxy"
Jul 07 12:04:17.131375 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:17.131356    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp"
Jul 07 12:04:17.143183 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:17.143161    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-068f74-openshift-default-dockercfg-grfzq\""
Jul 07 12:04:17.160238 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:17.160188    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp"]
Jul 07 12:04:17.208409 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:17.208367    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17652d42-c3a2-4a55-aa49-08067583e9a8-istio-token\") pod \"e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp\" (UID: \"17652d42-c3a2-4a55-aa49-08067583e9a8\") " pod="openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp"
Jul 07 12:04:17.208585 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:17.208416    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17652d42-c3a2-4a55-aa49-08067583e9a8-credential-socket\") pod \"e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp\" (UID: \"17652d42-c3a2-4a55-aa49-08067583e9a8\") " pod="openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp"
Jul 07 12:04:17.208585 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:17.208469    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17652d42-c3a2-4a55-aa49-08067583e9a8-workload-certs\") pod \"e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp\" (UID: \"17652d42-c3a2-4a55-aa49-08067583e9a8\") " pod="openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp"
Jul 07 12:04:17.208585 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:17.208529    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17652d42-c3a2-4a55-aa49-08067583e9a8-istio-data\") pod \"e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp\" (UID: \"17652d42-c3a2-4a55-aa49-08067583e9a8\") " pod="openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp"
Jul 07 12:04:17.208692 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:17.208643    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17652d42-c3a2-4a55-aa49-08067583e9a8-istiod-ca-cert\") pod \"e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp\" (UID: \"17652d42-c3a2-4a55-aa49-08067583e9a8\") " pod="openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp"
Jul 07 12:04:17.208692 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:17.208676    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17652d42-c3a2-4a55-aa49-08067583e9a8-workload-socket\") pod \"e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp\" (UID: \"17652d42-c3a2-4a55-aa49-08067583e9a8\") " pod="openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp"
Jul 07 12:04:17.208766 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:17.208700    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wjlz\" (UniqueName: \"kubernetes.io/projected/17652d42-c3a2-4a55-aa49-08067583e9a8-kube-api-access-8wjlz\") pod \"e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp\" (UID: \"17652d42-c3a2-4a55-aa49-08067583e9a8\") " pod="openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp"
Jul 07 12:04:17.208801 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:17.208771    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17652d42-c3a2-4a55-aa49-08067583e9a8-istio-envoy\") pod \"e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp\" (UID: \"17652d42-c3a2-4a55-aa49-08067583e9a8\") " pod="openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp"
Jul 07 12:04:17.208836 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:17.208819    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17652d42-c3a2-4a55-aa49-08067583e9a8-istio-podinfo\") pod \"e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp\" (UID: \"17652d42-c3a2-4a55-aa49-08067583e9a8\") " pod="openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp"
Jul 07 12:04:17.309528 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:17.309485    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17652d42-c3a2-4a55-aa49-08067583e9a8-istiod-ca-cert\") pod \"e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp\" (UID: \"17652d42-c3a2-4a55-aa49-08067583e9a8\") " pod="openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp"
Jul 07 12:04:17.309528 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:17.309530    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17652d42-c3a2-4a55-aa49-08067583e9a8-workload-socket\") pod \"e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp\" (UID: \"17652d42-c3a2-4a55-aa49-08067583e9a8\") " pod="openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp"
Jul 07 12:04:17.309798 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:17.309546    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wjlz\" (UniqueName: \"kubernetes.io/projected/17652d42-c3a2-4a55-aa49-08067583e9a8-kube-api-access-8wjlz\") pod \"e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp\" (UID: \"17652d42-c3a2-4a55-aa49-08067583e9a8\") " pod="openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp"
Jul 07 12:04:17.309798 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:17.309595    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17652d42-c3a2-4a55-aa49-08067583e9a8-istio-envoy\") pod \"e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp\" (UID: \"17652d42-c3a2-4a55-aa49-08067583e9a8\") " pod="openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp"
Jul 07 12:04:17.309798 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:17.309621    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17652d42-c3a2-4a55-aa49-08067583e9a8-istio-podinfo\") pod \"e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp\" (UID: \"17652d42-c3a2-4a55-aa49-08067583e9a8\") " pod="openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp"
Jul 07 12:04:17.309798 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:17.309670    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17652d42-c3a2-4a55-aa49-08067583e9a8-istio-token\") pod \"e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp\" (UID: \"17652d42-c3a2-4a55-aa49-08067583e9a8\") " pod="openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp"
Jul 07 12:04:17.309798 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:17.309701    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17652d42-c3a2-4a55-aa49-08067583e9a8-credential-socket\") pod \"e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp\" (UID: \"17652d42-c3a2-4a55-aa49-08067583e9a8\") " pod="openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp"
Jul 07 12:04:17.309798 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:17.309728    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17652d42-c3a2-4a55-aa49-08067583e9a8-workload-certs\") pod \"e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp\" (UID: \"17652d42-c3a2-4a55-aa49-08067583e9a8\") " pod="openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp"
Jul 07 12:04:17.309798 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:17.309767    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17652d42-c3a2-4a55-aa49-08067583e9a8-istio-data\") pod \"e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp\" (UID: \"17652d42-c3a2-4a55-aa49-08067583e9a8\") " pod="openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp"
Jul 07 12:04:17.310258 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:17.310008    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17652d42-c3a2-4a55-aa49-08067583e9a8-workload-socket\") pod \"e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp\" (UID: \"17652d42-c3a2-4a55-aa49-08067583e9a8\") " pod="openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp"
Jul 07 12:04:17.310258 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:17.310148    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17652d42-c3a2-4a55-aa49-08067583e9a8-istio-data\") pod \"e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp\" (UID: \"17652d42-c3a2-4a55-aa49-08067583e9a8\") " pod="openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp"
Jul 07 12:04:17.310385 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:17.310270    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17652d42-c3a2-4a55-aa49-08067583e9a8-credential-socket\") pod \"e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp\" (UID: \"17652d42-c3a2-4a55-aa49-08067583e9a8\") " pod="openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp"
Jul 07 12:04:17.310385 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:17.310318    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17652d42-c3a2-4a55-aa49-08067583e9a8-workload-certs\") pod \"e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp\" (UID: \"17652d42-c3a2-4a55-aa49-08067583e9a8\") " pod="openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp"
Jul 07 12:04:17.310385 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:17.310338    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17652d42-c3a2-4a55-aa49-08067583e9a8-istiod-ca-cert\") pod \"e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp\" (UID: \"17652d42-c3a2-4a55-aa49-08067583e9a8\") " pod="openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp"
Jul 07 12:04:17.311918 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:17.311895    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17652d42-c3a2-4a55-aa49-08067583e9a8-istio-envoy\") pod \"e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp\" (UID: \"17652d42-c3a2-4a55-aa49-08067583e9a8\") " pod="openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp"
Jul 07 12:04:17.312302 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:17.312285    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17652d42-c3a2-4a55-aa49-08067583e9a8-istio-podinfo\") pod \"e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp\" (UID: \"17652d42-c3a2-4a55-aa49-08067583e9a8\") " pod="openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp"
Jul 07 12:04:17.318159 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:17.318136    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17652d42-c3a2-4a55-aa49-08067583e9a8-istio-token\") pod \"e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp\" (UID: \"17652d42-c3a2-4a55-aa49-08067583e9a8\") " pod="openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp"
Jul 07 12:04:17.318255 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:17.318242    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wjlz\" (UniqueName: \"kubernetes.io/projected/17652d42-c3a2-4a55-aa49-08067583e9a8-kube-api-access-8wjlz\") pod \"e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp\" (UID: \"17652d42-c3a2-4a55-aa49-08067583e9a8\") " pod="openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp"
Jul 07 12:04:17.442420 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:17.442381    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp"
Jul 07 12:04:17.589943 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:17.589909    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp"]
Jul 07 12:04:17.590968 ip-10-0-135-78 kubenswrapper[2570]: W0707 12:04:17.590944    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17652d42_c3a2_4a55_aa49_08067583e9a8.slice/crio-c5bd165c84617ed1a84ad59c2127921ff948db1f50ba1899b03bbce4915096c7 WatchSource:0}: Error finding container c5bd165c84617ed1a84ad59c2127921ff948db1f50ba1899b03bbce4915096c7: Status 404 returned error can't find the container with id c5bd165c84617ed1a84ad59c2127921ff948db1f50ba1899b03bbce4915096c7
Jul 07 12:04:17.593118 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:17.593080    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 12:04:17.593180 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:17.593165    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 12:04:17.593262 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:17.593223    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 12:04:17.747158 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:17.747054    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp" event={"ID":"17652d42-c3a2-4a55-aa49-08067583e9a8","Type":"ContainerStarted","Data":"f9b3e5829928345ddfac5463c2242b07c6d33ab8a83190045614c5428f44b447"}
Jul 07 12:04:17.747158 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:17.747117    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp" event={"ID":"17652d42-c3a2-4a55-aa49-08067583e9a8","Type":"ContainerStarted","Data":"c5bd165c84617ed1a84ad59c2127921ff948db1f50ba1899b03bbce4915096c7"}
Jul 07 12:04:17.768160 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:17.768101    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp" podStartSLOduration=0.768083651 podStartE2EDuration="768.083651ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:04:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:04:17.76615147 +0000 UTC m=+2245.739791905" watchObservedRunningTime="2026-07-07 12:04:17.768083651 +0000 UTC m=+2245.741724060"
Jul 07 12:04:18.443318 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:18.443269    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp"
Jul 07 12:04:18.447999 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:18.447976    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp"
Jul 07 12:04:18.752597 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:18.752496    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp"
Jul 07 12:04:18.753424 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:18.753403    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp"
Jul 07 12:04:27.303323 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:27.303286    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-10f898/auth-test-model-10f898-kserve-6cb978c659-29qhs"]
Jul 07 12:04:27.303807 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:27.303751    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="5bccca9d-1571-45c6-8164-45f0c6648bd9" containerName="istio-proxy"
Jul 07 12:04:27.306806 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:27.306788    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-10f898/auth-test-model-10f898-kserve-6cb978c659-29qhs"
Jul 07 12:04:27.310920 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:27.310889    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-10f898\"/\"openshift-service-ca.crt\""
Jul 07 12:04:27.311102 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:27.310953    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-10f898\"/\"default-dockercfg-265ks\""
Jul 07 12:04:27.311102 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:27.311026    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-10f898\"/\"auth-test-model-10f898-kserve-self-signed-certs\""
Jul 07 12:04:27.311102 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:27.310890    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-10f898\"/\"kube-root-ca.crt\""
Jul 07 12:04:27.318274 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:27.318246    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-10f898/auth-test-model-10f898-kserve-6cb978c659-29qhs"]
Jul 07 12:04:27.408094 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:27.408058    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fde728ff-967a-4629-b9be-5dd4bab42fcf-tls-certs\") pod \"auth-test-model-10f898-kserve-6cb978c659-29qhs\" (UID: \"fde728ff-967a-4629-b9be-5dd4bab42fcf\") " pod="ai-tenant-e2e-shared-a-10f898/auth-test-model-10f898-kserve-6cb978c659-29qhs"
Jul 07 12:04:27.408094 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:27.408094    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brhvl\" (UniqueName: \"kubernetes.io/projected/fde728ff-967a-4629-b9be-5dd4bab42fcf-kube-api-access-brhvl\") pod \"auth-test-model-10f898-kserve-6cb978c659-29qhs\" (UID: \"fde728ff-967a-4629-b9be-5dd4bab42fcf\") " pod="ai-tenant-e2e-shared-a-10f898/auth-test-model-10f898-kserve-6cb978c659-29qhs"
Jul 07 12:04:27.408365 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:27.408131    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fde728ff-967a-4629-b9be-5dd4bab42fcf-model-cache\") pod \"auth-test-model-10f898-kserve-6cb978c659-29qhs\" (UID: \"fde728ff-967a-4629-b9be-5dd4bab42fcf\") " pod="ai-tenant-e2e-shared-a-10f898/auth-test-model-10f898-kserve-6cb978c659-29qhs"
Jul 07 12:04:27.408365 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:27.408180    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fde728ff-967a-4629-b9be-5dd4bab42fcf-home\") pod \"auth-test-model-10f898-kserve-6cb978c659-29qhs\" (UID: \"fde728ff-967a-4629-b9be-5dd4bab42fcf\") " pod="ai-tenant-e2e-shared-a-10f898/auth-test-model-10f898-kserve-6cb978c659-29qhs"
Jul 07 12:04:27.408365 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:27.408270    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fde728ff-967a-4629-b9be-5dd4bab42fcf-dshm\") pod \"auth-test-model-10f898-kserve-6cb978c659-29qhs\" (UID: \"fde728ff-967a-4629-b9be-5dd4bab42fcf\") " pod="ai-tenant-e2e-shared-a-10f898/auth-test-model-10f898-kserve-6cb978c659-29qhs"
Jul 07 12:04:27.408365 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:27.408308    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fde728ff-967a-4629-b9be-5dd4bab42fcf-tmp-dir\") pod \"auth-test-model-10f898-kserve-6cb978c659-29qhs\" (UID: \"fde728ff-967a-4629-b9be-5dd4bab42fcf\") " pod="ai-tenant-e2e-shared-a-10f898/auth-test-model-10f898-kserve-6cb978c659-29qhs"
Jul 07 12:04:27.508847 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:27.508802    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fde728ff-967a-4629-b9be-5dd4bab42fcf-tmp-dir\") pod \"auth-test-model-10f898-kserve-6cb978c659-29qhs\" (UID: \"fde728ff-967a-4629-b9be-5dd4bab42fcf\") " pod="ai-tenant-e2e-shared-a-10f898/auth-test-model-10f898-kserve-6cb978c659-29qhs"
Jul 07 12:04:27.509055 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:27.508897    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fde728ff-967a-4629-b9be-5dd4bab42fcf-tls-certs\") pod \"auth-test-model-10f898-kserve-6cb978c659-29qhs\" (UID: \"fde728ff-967a-4629-b9be-5dd4bab42fcf\") " pod="ai-tenant-e2e-shared-a-10f898/auth-test-model-10f898-kserve-6cb978c659-29qhs"
Jul 07 12:04:27.509055 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:27.508924    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brhvl\" (UniqueName: \"kubernetes.io/projected/fde728ff-967a-4629-b9be-5dd4bab42fcf-kube-api-access-brhvl\") pod \"auth-test-model-10f898-kserve-6cb978c659-29qhs\" (UID: \"fde728ff-967a-4629-b9be-5dd4bab42fcf\") " pod="ai-tenant-e2e-shared-a-10f898/auth-test-model-10f898-kserve-6cb978c659-29qhs"
Jul 07 12:04:27.509055 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:27.508969    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fde728ff-967a-4629-b9be-5dd4bab42fcf-model-cache\") pod \"auth-test-model-10f898-kserve-6cb978c659-29qhs\" (UID: \"fde728ff-967a-4629-b9be-5dd4bab42fcf\") " pod="ai-tenant-e2e-shared-a-10f898/auth-test-model-10f898-kserve-6cb978c659-29qhs"
Jul 07 12:04:27.509055 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:27.508994    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fde728ff-967a-4629-b9be-5dd4bab42fcf-home\") pod \"auth-test-model-10f898-kserve-6cb978c659-29qhs\" (UID: \"fde728ff-967a-4629-b9be-5dd4bab42fcf\") " pod="ai-tenant-e2e-shared-a-10f898/auth-test-model-10f898-kserve-6cb978c659-29qhs"
Jul 07 12:04:27.509325 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:27.509057    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fde728ff-967a-4629-b9be-5dd4bab42fcf-dshm\") pod \"auth-test-model-10f898-kserve-6cb978c659-29qhs\" (UID: \"fde728ff-967a-4629-b9be-5dd4bab42fcf\") " pod="ai-tenant-e2e-shared-a-10f898/auth-test-model-10f898-kserve-6cb978c659-29qhs"
Jul 07 12:04:27.509380 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:27.509356    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fde728ff-967a-4629-b9be-5dd4bab42fcf-tmp-dir\") pod \"auth-test-model-10f898-kserve-6cb978c659-29qhs\" (UID: \"fde728ff-967a-4629-b9be-5dd4bab42fcf\") " pod="ai-tenant-e2e-shared-a-10f898/auth-test-model-10f898-kserve-6cb978c659-29qhs"
Jul 07 12:04:27.509437 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:27.509413    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fde728ff-967a-4629-b9be-5dd4bab42fcf-model-cache\") pod \"auth-test-model-10f898-kserve-6cb978c659-29qhs\" (UID: \"fde728ff-967a-4629-b9be-5dd4bab42fcf\") " pod="ai-tenant-e2e-shared-a-10f898/auth-test-model-10f898-kserve-6cb978c659-29qhs"
Jul 07 12:04:27.509474 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:27.509445    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fde728ff-967a-4629-b9be-5dd4bab42fcf-home\") pod \"auth-test-model-10f898-kserve-6cb978c659-29qhs\" (UID: \"fde728ff-967a-4629-b9be-5dd4bab42fcf\") " pod="ai-tenant-e2e-shared-a-10f898/auth-test-model-10f898-kserve-6cb978c659-29qhs"
Jul 07 12:04:27.511372 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:27.511346    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fde728ff-967a-4629-b9be-5dd4bab42fcf-dshm\") pod \"auth-test-model-10f898-kserve-6cb978c659-29qhs\" (UID: \"fde728ff-967a-4629-b9be-5dd4bab42fcf\") " pod="ai-tenant-e2e-shared-a-10f898/auth-test-model-10f898-kserve-6cb978c659-29qhs"
Jul 07 12:04:27.511542 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:27.511525    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fde728ff-967a-4629-b9be-5dd4bab42fcf-tls-certs\") pod \"auth-test-model-10f898-kserve-6cb978c659-29qhs\" (UID: \"fde728ff-967a-4629-b9be-5dd4bab42fcf\") " pod="ai-tenant-e2e-shared-a-10f898/auth-test-model-10f898-kserve-6cb978c659-29qhs"
Jul 07 12:04:27.517665 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:27.517636    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-brhvl\" (UniqueName: \"kubernetes.io/projected/fde728ff-967a-4629-b9be-5dd4bab42fcf-kube-api-access-brhvl\") pod \"auth-test-model-10f898-kserve-6cb978c659-29qhs\" (UID: \"fde728ff-967a-4629-b9be-5dd4bab42fcf\") " pod="ai-tenant-e2e-shared-a-10f898/auth-test-model-10f898-kserve-6cb978c659-29qhs"
Jul 07 12:04:27.619538 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:27.619445    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-10f898/auth-test-model-10f898-kserve-6cb978c659-29qhs"
Jul 07 12:04:27.795128 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:27.795103    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-10f898/auth-test-model-10f898-kserve-6cb978c659-29qhs"]
Jul 07 12:04:27.797276 ip-10-0-135-78 kubenswrapper[2570]: W0707 12:04:27.797245    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfde728ff_967a_4629_b9be_5dd4bab42fcf.slice/crio-bac04f26df36e75c97a3a8b9994767249e6a2da3ae84dc28b59aba656f5e2e01 WatchSource:0}: Error finding container bac04f26df36e75c97a3a8b9994767249e6a2da3ae84dc28b59aba656f5e2e01: Status 404 returned error can't find the container with id bac04f26df36e75c97a3a8b9994767249e6a2da3ae84dc28b59aba656f5e2e01
Jul 07 12:04:28.806693 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:28.806656    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-10f898/auth-test-model-10f898-kserve-6cb978c659-29qhs" event={"ID":"fde728ff-967a-4629-b9be-5dd4bab42fcf","Type":"ContainerStarted","Data":"14a65e002933f8e0ca5ee4d38efc944eee489c9d6312e0d9f650de23881fdf5a"}
Jul 07 12:04:28.806693 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:28.806698    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-10f898/auth-test-model-10f898-kserve-6cb978c659-29qhs" event={"ID":"fde728ff-967a-4629-b9be-5dd4bab42fcf","Type":"ContainerStarted","Data":"bac04f26df36e75c97a3a8b9994767249e6a2da3ae84dc28b59aba656f5e2e01"}
Jul 07 12:04:33.754309 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:33.754263    2570 patch_prober.go:28] interesting pod/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Jul 07 12:04:33.754772 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:33.754332    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp" podUID="17652d42-c3a2-4a55-aa49-08067583e9a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Jul 07 12:04:33.843188 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:33.843150    2570 generic.go:356] "Generic (PLEG): container finished" podID="17652d42-c3a2-4a55-aa49-08067583e9a8" containerID="f9b3e5829928345ddfac5463c2242b07c6d33ab8a83190045614c5428f44b447" exitCode=0
Jul 07 12:04:33.843388 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:33.843249    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp" event={"ID":"17652d42-c3a2-4a55-aa49-08067583e9a8","Type":"ContainerDied","Data":"f9b3e5829928345ddfac5463c2242b07c6d33ab8a83190045614c5428f44b447"}
Jul 07 12:04:33.843722 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:33.843692    2570 scope.go:117] "RemoveContainer" containerID="f9b3e5829928345ddfac5463c2242b07c6d33ab8a83190045614c5428f44b447"
Jul 07 12:04:33.844743 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:33.844715    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 12:04:33.844843 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:33.844784    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 12:04:33.844843 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:33.844815    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 12:04:33.865047 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:33.865001    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-10f898/auth-test-model-10f898-kserve-6cb978c659-29qhs" podStartSLOduration=6.864981311 podStartE2EDuration="6.864981311s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:04:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:04:28.833749557 +0000 UTC m=+2256.807389990" watchObservedRunningTime="2026-07-07 12:04:33.864981311 +0000 UTC m=+2261.838621721"
Jul 07 12:04:34.851118 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:34.851085    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp" event={"ID":"17652d42-c3a2-4a55-aa49-08067583e9a8","Type":"ContainerStarted","Data":"098f6a4313b21a43fa7527cfb3feff541e1c8722f3591d2903cd30ab3f06c61b"}
Jul 07 12:04:35.443419 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:35.443373    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp"
Jul 07 12:04:35.443647 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:35.443524    2570 patch_prober.go:28] interesting pod/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Jul 07 12:04:35.443647 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:35.443568    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp" podUID="17652d42-c3a2-4a55-aa49-08067583e9a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Jul 07 12:04:36.443125 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:36.443084    2570 patch_prober.go:28] interesting pod/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Jul 07 12:04:36.443764 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:36.443727    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp" podUID="17652d42-c3a2-4a55-aa49-08067583e9a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Jul 07 12:04:37.443341 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:37.443300    2570 patch_prober.go:28] interesting pod/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Jul 07 12:04:37.443954 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:37.443922    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp" podUID="17652d42-c3a2-4a55-aa49-08067583e9a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Jul 07 12:04:37.619692 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:37.619656    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-10f898/auth-test-model-10f898-kserve-6cb978c659-29qhs"
Jul 07 12:04:37.619875 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:37.619722    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-10f898/auth-test-model-10f898-kserve-6cb978c659-29qhs"
Jul 07 12:04:37.654052 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:37.654022    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-10f898/auth-test-model-10f898-kserve-6cb978c659-29qhs"
Jul 07 12:04:37.882903 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:37.882809    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-10f898/auth-test-model-10f898-kserve-6cb978c659-29qhs"
Jul 07 12:04:38.443875 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:38.443833    2570 patch_prober.go:28] interesting pod/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Jul 07 12:04:38.444505 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:38.444469    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp" podUID="17652d42-c3a2-4a55-aa49-08067583e9a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Jul 07 12:04:39.447230 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:39.447183    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp"
Jul 07 12:04:39.447692 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:39.447675    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp"
Jul 07 12:04:39.448337 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:39.448321    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp"
Jul 07 12:04:44.313902 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:44.313862    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-068f74/auth-test-model-068f74-kserve-5874887fcf-n6cwg"]
Jul 07 12:04:44.316366 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:44.316348    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-068f74/auth-test-model-068f74-kserve-5874887fcf-n6cwg"
Jul 07 12:04:44.320483 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:44.320298    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-068f74\"/\"openshift-service-ca.crt\""
Jul 07 12:04:44.320483 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:44.320335    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-068f74\"/\"auth-test-model-068f74-kserve-self-signed-certs\""
Jul 07 12:04:44.320483 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:44.320346    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-068f74\"/\"kube-root-ca.crt\""
Jul 07 12:04:44.320483 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:44.320351    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-068f74\"/\"default-dockercfg-ttmbl\""
Jul 07 12:04:44.330777 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:44.330749    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-068f74/auth-test-model-068f74-kserve-5874887fcf-n6cwg"]
Jul 07 12:04:44.378513 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:44.378475    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8b262d9-19c8-48d6-a9d0-2e5701cb7c34-model-cache\") pod \"auth-test-model-068f74-kserve-5874887fcf-n6cwg\" (UID: \"a8b262d9-19c8-48d6-a9d0-2e5701cb7c34\") " pod="ai-tenant-e2e-shared-b-068f74/auth-test-model-068f74-kserve-5874887fcf-n6cwg"
Jul 07 12:04:44.378513 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:44.378521    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8b262d9-19c8-48d6-a9d0-2e5701cb7c34-tls-certs\") pod \"auth-test-model-068f74-kserve-5874887fcf-n6cwg\" (UID: \"a8b262d9-19c8-48d6-a9d0-2e5701cb7c34\") " pod="ai-tenant-e2e-shared-b-068f74/auth-test-model-068f74-kserve-5874887fcf-n6cwg"
Jul 07 12:04:44.378744 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:44.378554    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jvhk\" (UniqueName: \"kubernetes.io/projected/a8b262d9-19c8-48d6-a9d0-2e5701cb7c34-kube-api-access-5jvhk\") pod \"auth-test-model-068f74-kserve-5874887fcf-n6cwg\" (UID: \"a8b262d9-19c8-48d6-a9d0-2e5701cb7c34\") " pod="ai-tenant-e2e-shared-b-068f74/auth-test-model-068f74-kserve-5874887fcf-n6cwg"
Jul 07 12:04:44.378744 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:44.378598    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8b262d9-19c8-48d6-a9d0-2e5701cb7c34-tmp-dir\") pod \"auth-test-model-068f74-kserve-5874887fcf-n6cwg\" (UID: \"a8b262d9-19c8-48d6-a9d0-2e5701cb7c34\") " pod="ai-tenant-e2e-shared-b-068f74/auth-test-model-068f74-kserve-5874887fcf-n6cwg"
Jul 07 12:04:44.378840 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:44.378760    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8b262d9-19c8-48d6-a9d0-2e5701cb7c34-home\") pod \"auth-test-model-068f74-kserve-5874887fcf-n6cwg\" (UID: \"a8b262d9-19c8-48d6-a9d0-2e5701cb7c34\") " pod="ai-tenant-e2e-shared-b-068f74/auth-test-model-068f74-kserve-5874887fcf-n6cwg"
Jul 07 12:04:44.378840 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:44.378812    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8b262d9-19c8-48d6-a9d0-2e5701cb7c34-dshm\") pod \"auth-test-model-068f74-kserve-5874887fcf-n6cwg\" (UID: \"a8b262d9-19c8-48d6-a9d0-2e5701cb7c34\") " pod="ai-tenant-e2e-shared-b-068f74/auth-test-model-068f74-kserve-5874887fcf-n6cwg"
Jul 07 12:04:44.479984 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:44.479931    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8b262d9-19c8-48d6-a9d0-2e5701cb7c34-home\") pod \"auth-test-model-068f74-kserve-5874887fcf-n6cwg\" (UID: \"a8b262d9-19c8-48d6-a9d0-2e5701cb7c34\") " pod="ai-tenant-e2e-shared-b-068f74/auth-test-model-068f74-kserve-5874887fcf-n6cwg"
Jul 07 12:04:44.479984 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:44.479977    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8b262d9-19c8-48d6-a9d0-2e5701cb7c34-dshm\") pod \"auth-test-model-068f74-kserve-5874887fcf-n6cwg\" (UID: \"a8b262d9-19c8-48d6-a9d0-2e5701cb7c34\") " pod="ai-tenant-e2e-shared-b-068f74/auth-test-model-068f74-kserve-5874887fcf-n6cwg"
Jul 07 12:04:44.480297 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:44.480015    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8b262d9-19c8-48d6-a9d0-2e5701cb7c34-model-cache\") pod \"auth-test-model-068f74-kserve-5874887fcf-n6cwg\" (UID: \"a8b262d9-19c8-48d6-a9d0-2e5701cb7c34\") " pod="ai-tenant-e2e-shared-b-068f74/auth-test-model-068f74-kserve-5874887fcf-n6cwg"
Jul 07 12:04:44.480297 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:44.480037    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8b262d9-19c8-48d6-a9d0-2e5701cb7c34-tls-certs\") pod \"auth-test-model-068f74-kserve-5874887fcf-n6cwg\" (UID: \"a8b262d9-19c8-48d6-a9d0-2e5701cb7c34\") " pod="ai-tenant-e2e-shared-b-068f74/auth-test-model-068f74-kserve-5874887fcf-n6cwg"
Jul 07 12:04:44.480297 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:44.480056    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jvhk\" (UniqueName: \"kubernetes.io/projected/a8b262d9-19c8-48d6-a9d0-2e5701cb7c34-kube-api-access-5jvhk\") pod \"auth-test-model-068f74-kserve-5874887fcf-n6cwg\" (UID: \"a8b262d9-19c8-48d6-a9d0-2e5701cb7c34\") " pod="ai-tenant-e2e-shared-b-068f74/auth-test-model-068f74-kserve-5874887fcf-n6cwg"
Jul 07 12:04:44.480297 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:44.480236    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8b262d9-19c8-48d6-a9d0-2e5701cb7c34-tmp-dir\") pod \"auth-test-model-068f74-kserve-5874887fcf-n6cwg\" (UID: \"a8b262d9-19c8-48d6-a9d0-2e5701cb7c34\") " pod="ai-tenant-e2e-shared-b-068f74/auth-test-model-068f74-kserve-5874887fcf-n6cwg"
Jul 07 12:04:44.480507 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:44.480404    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8b262d9-19c8-48d6-a9d0-2e5701cb7c34-home\") pod \"auth-test-model-068f74-kserve-5874887fcf-n6cwg\" (UID: \"a8b262d9-19c8-48d6-a9d0-2e5701cb7c34\") " pod="ai-tenant-e2e-shared-b-068f74/auth-test-model-068f74-kserve-5874887fcf-n6cwg"
Jul 07 12:04:44.480636 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:44.480612    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8b262d9-19c8-48d6-a9d0-2e5701cb7c34-tmp-dir\") pod \"auth-test-model-068f74-kserve-5874887fcf-n6cwg\" (UID: \"a8b262d9-19c8-48d6-a9d0-2e5701cb7c34\") " pod="ai-tenant-e2e-shared-b-068f74/auth-test-model-068f74-kserve-5874887fcf-n6cwg"
Jul 07 12:04:44.480692 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:44.480624    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8b262d9-19c8-48d6-a9d0-2e5701cb7c34-model-cache\") pod \"auth-test-model-068f74-kserve-5874887fcf-n6cwg\" (UID: \"a8b262d9-19c8-48d6-a9d0-2e5701cb7c34\") " pod="ai-tenant-e2e-shared-b-068f74/auth-test-model-068f74-kserve-5874887fcf-n6cwg"
Jul 07 12:04:44.482339 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:44.482310    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8b262d9-19c8-48d6-a9d0-2e5701cb7c34-dshm\") pod \"auth-test-model-068f74-kserve-5874887fcf-n6cwg\" (UID: \"a8b262d9-19c8-48d6-a9d0-2e5701cb7c34\") " pod="ai-tenant-e2e-shared-b-068f74/auth-test-model-068f74-kserve-5874887fcf-n6cwg"
Jul 07 12:04:44.482683 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:44.482656    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8b262d9-19c8-48d6-a9d0-2e5701cb7c34-tls-certs\") pod \"auth-test-model-068f74-kserve-5874887fcf-n6cwg\" (UID: \"a8b262d9-19c8-48d6-a9d0-2e5701cb7c34\") " pod="ai-tenant-e2e-shared-b-068f74/auth-test-model-068f74-kserve-5874887fcf-n6cwg"
Jul 07 12:04:44.494070 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:44.494043    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jvhk\" (UniqueName: \"kubernetes.io/projected/a8b262d9-19c8-48d6-a9d0-2e5701cb7c34-kube-api-access-5jvhk\") pod \"auth-test-model-068f74-kserve-5874887fcf-n6cwg\" (UID: \"a8b262d9-19c8-48d6-a9d0-2e5701cb7c34\") " pod="ai-tenant-e2e-shared-b-068f74/auth-test-model-068f74-kserve-5874887fcf-n6cwg"
Jul 07 12:04:44.627107 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:44.627016    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-068f74/auth-test-model-068f74-kserve-5874887fcf-n6cwg"
Jul 07 12:04:44.769567 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:44.769533    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-068f74/auth-test-model-068f74-kserve-5874887fcf-n6cwg"]
Jul 07 12:04:44.770356 ip-10-0-135-78 kubenswrapper[2570]: W0707 12:04:44.770329    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda8b262d9_19c8_48d6_a9d0_2e5701cb7c34.slice/crio-78d4881ed8b41833d438fa984d488c052b9b31569a29c82653290255a7b62265 WatchSource:0}: Error finding container 78d4881ed8b41833d438fa984d488c052b9b31569a29c82653290255a7b62265: Status 404 returned error can't find the container with id 78d4881ed8b41833d438fa984d488c052b9b31569a29c82653290255a7b62265
Jul 07 12:04:44.911778 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:44.911678    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-068f74/auth-test-model-068f74-kserve-5874887fcf-n6cwg" event={"ID":"a8b262d9-19c8-48d6-a9d0-2e5701cb7c34","Type":"ContainerStarted","Data":"60188fb67ac08f1bed376ea5df1b4a80db12ce568d5db4373211dfd5f6fa7734"}
Jul 07 12:04:44.911778 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:44.911730    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-068f74/auth-test-model-068f74-kserve-5874887fcf-n6cwg" event={"ID":"a8b262d9-19c8-48d6-a9d0-2e5701cb7c34","Type":"ContainerStarted","Data":"78d4881ed8b41833d438fa984d488c052b9b31569a29c82653290255a7b62265"}
Jul 07 12:04:44.930606 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:44.930556    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-068f74/auth-test-model-068f74-kserve-5874887fcf-n6cwg" podStartSLOduration=0.930536405 podStartE2EDuration="930.536405ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:04:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:04:44.929818415 +0000 UTC m=+2272.903458825" watchObservedRunningTime="2026-07-07 12:04:44.930536405 +0000 UTC m=+2272.904176817"
Jul 07 12:04:54.627151 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:54.627108    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-068f74/auth-test-model-068f74-kserve-5874887fcf-n6cwg"
Jul 07 12:04:54.627627 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:54.627192    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-068f74/auth-test-model-068f74-kserve-5874887fcf-n6cwg"
Jul 07 12:04:54.639674 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:54.639646    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-068f74/auth-test-model-068f74-kserve-5874887fcf-n6cwg"
Jul 07 12:04:54.980693 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:04:54.980661    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-068f74/auth-test-model-068f74-kserve-5874887fcf-n6cwg"
Jul 07 12:06:36.008303 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:36.008270    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-10f898/auth-test-model-10f898-kserve-6cb978c659-29qhs"]
Jul 07 12:06:36.997901 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:36.997865    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-068f74/auth-test-model-068f74-kserve-5874887fcf-n6cwg"]
Jul 07 12:06:46.415998 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:46.415923    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-10f898/sub-test-model-10f898-kserve-7cd676f44c-d9lqg"]
Jul 07 12:06:46.418327 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:46.418310    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-10f898/sub-test-model-10f898-kserve-7cd676f44c-d9lqg"
Jul 07 12:06:46.421173 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:46.421150    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-10f898\"/\"sub-test-model-10f898-kserve-self-signed-certs\""
Jul 07 12:06:46.428475 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:46.428451    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-10f898/sub-test-model-10f898-kserve-7cd676f44c-d9lqg"]
Jul 07 12:06:46.452679 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:46.452649    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4ksn\" (UniqueName: \"kubernetes.io/projected/bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a-kube-api-access-f4ksn\") pod \"sub-test-model-10f898-kserve-7cd676f44c-d9lqg\" (UID: \"bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a\") " pod="ai-tenant-e2e-shared-a-10f898/sub-test-model-10f898-kserve-7cd676f44c-d9lqg"
Jul 07 12:06:46.452813 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:46.452688    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a-tls-certs\") pod \"sub-test-model-10f898-kserve-7cd676f44c-d9lqg\" (UID: \"bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a\") " pod="ai-tenant-e2e-shared-a-10f898/sub-test-model-10f898-kserve-7cd676f44c-d9lqg"
Jul 07 12:06:46.452813 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:46.452709    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a-home\") pod \"sub-test-model-10f898-kserve-7cd676f44c-d9lqg\" (UID: \"bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a\") " pod="ai-tenant-e2e-shared-a-10f898/sub-test-model-10f898-kserve-7cd676f44c-d9lqg"
Jul 07 12:06:46.452813 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:46.452783    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a-dshm\") pod \"sub-test-model-10f898-kserve-7cd676f44c-d9lqg\" (UID: \"bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a\") " pod="ai-tenant-e2e-shared-a-10f898/sub-test-model-10f898-kserve-7cd676f44c-d9lqg"
Jul 07 12:06:46.452813 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:46.452806    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a-model-cache\") pod \"sub-test-model-10f898-kserve-7cd676f44c-d9lqg\" (UID: \"bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a\") " pod="ai-tenant-e2e-shared-a-10f898/sub-test-model-10f898-kserve-7cd676f44c-d9lqg"
Jul 07 12:06:46.452953 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:46.452875    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a-tmp-dir\") pod \"sub-test-model-10f898-kserve-7cd676f44c-d9lqg\" (UID: \"bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a\") " pod="ai-tenant-e2e-shared-a-10f898/sub-test-model-10f898-kserve-7cd676f44c-d9lqg"
Jul 07 12:06:46.554154 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:46.554119    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a-dshm\") pod \"sub-test-model-10f898-kserve-7cd676f44c-d9lqg\" (UID: \"bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a\") " pod="ai-tenant-e2e-shared-a-10f898/sub-test-model-10f898-kserve-7cd676f44c-d9lqg"
Jul 07 12:06:46.554154 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:46.554159    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a-model-cache\") pod \"sub-test-model-10f898-kserve-7cd676f44c-d9lqg\" (UID: \"bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a\") " pod="ai-tenant-e2e-shared-a-10f898/sub-test-model-10f898-kserve-7cd676f44c-d9lqg"
Jul 07 12:06:46.554396 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:46.554202    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a-tmp-dir\") pod \"sub-test-model-10f898-kserve-7cd676f44c-d9lqg\" (UID: \"bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a\") " pod="ai-tenant-e2e-shared-a-10f898/sub-test-model-10f898-kserve-7cd676f44c-d9lqg"
Jul 07 12:06:46.554396 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:46.554262    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4ksn\" (UniqueName: \"kubernetes.io/projected/bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a-kube-api-access-f4ksn\") pod \"sub-test-model-10f898-kserve-7cd676f44c-d9lqg\" (UID: \"bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a\") " pod="ai-tenant-e2e-shared-a-10f898/sub-test-model-10f898-kserve-7cd676f44c-d9lqg"
Jul 07 12:06:46.554493 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:46.554390    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a-tls-certs\") pod \"sub-test-model-10f898-kserve-7cd676f44c-d9lqg\" (UID: \"bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a\") " pod="ai-tenant-e2e-shared-a-10f898/sub-test-model-10f898-kserve-7cd676f44c-d9lqg"
Jul 07 12:06:46.554493 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:46.554458    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a-home\") pod \"sub-test-model-10f898-kserve-7cd676f44c-d9lqg\" (UID: \"bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a\") " pod="ai-tenant-e2e-shared-a-10f898/sub-test-model-10f898-kserve-7cd676f44c-d9lqg"
Jul 07 12:06:46.554653 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:46.554633    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a-model-cache\") pod \"sub-test-model-10f898-kserve-7cd676f44c-d9lqg\" (UID: \"bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a\") " pod="ai-tenant-e2e-shared-a-10f898/sub-test-model-10f898-kserve-7cd676f44c-d9lqg"
Jul 07 12:06:46.554728 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:46.554711    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a-tmp-dir\") pod \"sub-test-model-10f898-kserve-7cd676f44c-d9lqg\" (UID: \"bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a\") " pod="ai-tenant-e2e-shared-a-10f898/sub-test-model-10f898-kserve-7cd676f44c-d9lqg"
Jul 07 12:06:46.554830 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:46.554804    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a-home\") pod \"sub-test-model-10f898-kserve-7cd676f44c-d9lqg\" (UID: \"bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a\") " pod="ai-tenant-e2e-shared-a-10f898/sub-test-model-10f898-kserve-7cd676f44c-d9lqg"
Jul 07 12:06:46.556529 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:46.556500    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a-dshm\") pod \"sub-test-model-10f898-kserve-7cd676f44c-d9lqg\" (UID: \"bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a\") " pod="ai-tenant-e2e-shared-a-10f898/sub-test-model-10f898-kserve-7cd676f44c-d9lqg"
Jul 07 12:06:46.556839 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:46.556822    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a-tls-certs\") pod \"sub-test-model-10f898-kserve-7cd676f44c-d9lqg\" (UID: \"bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a\") " pod="ai-tenant-e2e-shared-a-10f898/sub-test-model-10f898-kserve-7cd676f44c-d9lqg"
Jul 07 12:06:46.562756 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:46.562738    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4ksn\" (UniqueName: \"kubernetes.io/projected/bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a-kube-api-access-f4ksn\") pod \"sub-test-model-10f898-kserve-7cd676f44c-d9lqg\" (UID: \"bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a\") " pod="ai-tenant-e2e-shared-a-10f898/sub-test-model-10f898-kserve-7cd676f44c-d9lqg"
Jul 07 12:06:46.728536 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:46.728483    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-10f898/sub-test-model-10f898-kserve-7cd676f44c-d9lqg"
Jul 07 12:06:46.863727 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:46.863698    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-10f898/sub-test-model-10f898-kserve-7cd676f44c-d9lqg"]
Jul 07 12:06:46.865770 ip-10-0-135-78 kubenswrapper[2570]: W0707 12:06:46.865741    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbdc1b5dc_b11c_47f8_a7e1_95c5d385df2a.slice/crio-ade36546bedef0b719aca04616c7f84e25733bcb178ce503aa5b680bd495bd4d WatchSource:0}: Error finding container ade36546bedef0b719aca04616c7f84e25733bcb178ce503aa5b680bd495bd4d: Status 404 returned error can't find the container with id ade36546bedef0b719aca04616c7f84e25733bcb178ce503aa5b680bd495bd4d
Jul 07 12:06:47.574185 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:47.574145    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-10f898/sub-test-model-10f898-kserve-7cd676f44c-d9lqg" event={"ID":"bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a","Type":"ContainerStarted","Data":"27b1343e5865161f684b8de28386ec9e8434f5214fe056ec8f9f05ef0d36df4e"}
Jul 07 12:06:47.574185 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:47.574191    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-10f898/sub-test-model-10f898-kserve-7cd676f44c-d9lqg" event={"ID":"bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a","Type":"ContainerStarted","Data":"ade36546bedef0b719aca04616c7f84e25733bcb178ce503aa5b680bd495bd4d"}
Jul 07 12:06:47.592422 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:47.592370    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-10f898/sub-test-model-10f898-kserve-7cd676f44c-d9lqg" podStartSLOduration=1.5923557929999999 podStartE2EDuration="1.592355793s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:06:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:06:47.590159282 +0000 UTC m=+2395.563799693" watchObservedRunningTime="2026-07-07 12:06:47.592355793 +0000 UTC m=+2395.565996209"
Jul 07 12:06:51.028570 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:51.028523    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-10f898/auth-test-model-10f898-kserve-6cb978c659-29qhs" podUID="fde728ff-967a-4629-b9be-5dd4bab42fcf" containerName="main" containerID="cri-o://14a65e002933f8e0ca5ee4d38efc944eee489c9d6312e0d9f650de23881fdf5a" gracePeriod=45
Jul 07 12:06:51.289034 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:51.288964    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-10f898/auth-test-model-10f898-kserve-6cb978c659-29qhs"
Jul 07 12:06:51.398519 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:51.398481    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fde728ff-967a-4629-b9be-5dd4bab42fcf-home\") pod \"fde728ff-967a-4629-b9be-5dd4bab42fcf\" (UID: \"fde728ff-967a-4629-b9be-5dd4bab42fcf\") "
Jul 07 12:06:51.398742 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:51.398555    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fde728ff-967a-4629-b9be-5dd4bab42fcf-tls-certs\") pod \"fde728ff-967a-4629-b9be-5dd4bab42fcf\" (UID: \"fde728ff-967a-4629-b9be-5dd4bab42fcf\") "
Jul 07 12:06:51.398742 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:51.398583    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-brhvl\" (UniqueName: \"kubernetes.io/projected/fde728ff-967a-4629-b9be-5dd4bab42fcf-kube-api-access-brhvl\") pod \"fde728ff-967a-4629-b9be-5dd4bab42fcf\" (UID: \"fde728ff-967a-4629-b9be-5dd4bab42fcf\") "
Jul 07 12:06:51.398742 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:51.398598    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fde728ff-967a-4629-b9be-5dd4bab42fcf-tmp-dir\") pod \"fde728ff-967a-4629-b9be-5dd4bab42fcf\" (UID: \"fde728ff-967a-4629-b9be-5dd4bab42fcf\") "
Jul 07 12:06:51.398742 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:51.398681    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fde728ff-967a-4629-b9be-5dd4bab42fcf-model-cache\") pod \"fde728ff-967a-4629-b9be-5dd4bab42fcf\" (UID: \"fde728ff-967a-4629-b9be-5dd4bab42fcf\") "
Jul 07 12:06:51.398742 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:51.398698    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fde728ff-967a-4629-b9be-5dd4bab42fcf-dshm\") pod \"fde728ff-967a-4629-b9be-5dd4bab42fcf\" (UID: \"fde728ff-967a-4629-b9be-5dd4bab42fcf\") "
Jul 07 12:06:51.399030 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:51.398822    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fde728ff-967a-4629-b9be-5dd4bab42fcf-home" (OuterVolumeSpecName: "home") pod "fde728ff-967a-4629-b9be-5dd4bab42fcf" (UID: "fde728ff-967a-4629-b9be-5dd4bab42fcf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:06:51.399030 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:51.398922    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fde728ff-967a-4629-b9be-5dd4bab42fcf-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fde728ff-967a-4629-b9be-5dd4bab42fcf" (UID: "fde728ff-967a-4629-b9be-5dd4bab42fcf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:06:51.399030 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:51.398946    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fde728ff-967a-4629-b9be-5dd4bab42fcf-model-cache" (OuterVolumeSpecName: "model-cache") pod "fde728ff-967a-4629-b9be-5dd4bab42fcf" (UID: "fde728ff-967a-4629-b9be-5dd4bab42fcf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:06:51.399030 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:51.398990    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fde728ff-967a-4629-b9be-5dd4bab42fcf-home\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:06:51.400906 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:51.400871    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fde728ff-967a-4629-b9be-5dd4bab42fcf-dshm" (OuterVolumeSpecName: "dshm") pod "fde728ff-967a-4629-b9be-5dd4bab42fcf" (UID: "fde728ff-967a-4629-b9be-5dd4bab42fcf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:06:51.401035 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:51.400946    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fde728ff-967a-4629-b9be-5dd4bab42fcf-kube-api-access-brhvl" (OuterVolumeSpecName: "kube-api-access-brhvl") pod "fde728ff-967a-4629-b9be-5dd4bab42fcf" (UID: "fde728ff-967a-4629-b9be-5dd4bab42fcf"). InnerVolumeSpecName "kube-api-access-brhvl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 12:06:51.401035 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:51.400962    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fde728ff-967a-4629-b9be-5dd4bab42fcf-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fde728ff-967a-4629-b9be-5dd4bab42fcf" (UID: "fde728ff-967a-4629-b9be-5dd4bab42fcf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 12:06:51.500299 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:51.500254    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fde728ff-967a-4629-b9be-5dd4bab42fcf-model-cache\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:06:51.500299 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:51.500292    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fde728ff-967a-4629-b9be-5dd4bab42fcf-dshm\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:06:51.500299 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:51.500303    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fde728ff-967a-4629-b9be-5dd4bab42fcf-tls-certs\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:06:51.500556 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:51.500316    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-brhvl\" (UniqueName: \"kubernetes.io/projected/fde728ff-967a-4629-b9be-5dd4bab42fcf-kube-api-access-brhvl\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:06:51.500556 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:51.500330    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fde728ff-967a-4629-b9be-5dd4bab42fcf-tmp-dir\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:06:51.596579 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:51.596477    2570 generic.go:356] "Generic (PLEG): container finished" podID="fde728ff-967a-4629-b9be-5dd4bab42fcf" containerID="14a65e002933f8e0ca5ee4d38efc944eee489c9d6312e0d9f650de23881fdf5a" exitCode=0
Jul 07 12:06:51.596579 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:51.596522    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-10f898/auth-test-model-10f898-kserve-6cb978c659-29qhs"
Jul 07 12:06:51.596775 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:51.596567    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-10f898/auth-test-model-10f898-kserve-6cb978c659-29qhs" event={"ID":"fde728ff-967a-4629-b9be-5dd4bab42fcf","Type":"ContainerDied","Data":"14a65e002933f8e0ca5ee4d38efc944eee489c9d6312e0d9f650de23881fdf5a"}
Jul 07 12:06:51.596775 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:51.596615    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-10f898/auth-test-model-10f898-kserve-6cb978c659-29qhs" event={"ID":"fde728ff-967a-4629-b9be-5dd4bab42fcf","Type":"ContainerDied","Data":"bac04f26df36e75c97a3a8b9994767249e6a2da3ae84dc28b59aba656f5e2e01"}
Jul 07 12:06:51.596775 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:51.596634    2570 scope.go:117] "RemoveContainer" containerID="14a65e002933f8e0ca5ee4d38efc944eee489c9d6312e0d9f650de23881fdf5a"
Jul 07 12:06:51.606287 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:51.606268    2570 scope.go:117] "RemoveContainer" containerID="14a65e002933f8e0ca5ee4d38efc944eee489c9d6312e0d9f650de23881fdf5a"
Jul 07 12:06:51.606570 ip-10-0-135-78 kubenswrapper[2570]: E0707 12:06:51.606543    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14a65e002933f8e0ca5ee4d38efc944eee489c9d6312e0d9f650de23881fdf5a\": container with ID starting with 14a65e002933f8e0ca5ee4d38efc944eee489c9d6312e0d9f650de23881fdf5a not found: ID does not exist" containerID="14a65e002933f8e0ca5ee4d38efc944eee489c9d6312e0d9f650de23881fdf5a"
Jul 07 12:06:51.606664 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:51.606578    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14a65e002933f8e0ca5ee4d38efc944eee489c9d6312e0d9f650de23881fdf5a"} err="failed to get container status \"14a65e002933f8e0ca5ee4d38efc944eee489c9d6312e0d9f650de23881fdf5a\": rpc error: code = NotFound desc = could not find container \"14a65e002933f8e0ca5ee4d38efc944eee489c9d6312e0d9f650de23881fdf5a\": container with ID starting with 14a65e002933f8e0ca5ee4d38efc944eee489c9d6312e0d9f650de23881fdf5a not found: ID does not exist"
Jul 07 12:06:51.618861 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:51.618836    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-10f898/auth-test-model-10f898-kserve-6cb978c659-29qhs"]
Jul 07 12:06:51.622134 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:51.622113    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-10f898/auth-test-model-10f898-kserve-6cb978c659-29qhs"]
Jul 07 12:06:51.944957 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:51.944918    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fde728ff-967a-4629-b9be-5dd4bab42fcf" path="/var/lib/kubelet/pods/fde728ff-967a-4629-b9be-5dd4bab42fcf/volumes"
Jul 07 12:06:52.018458 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:52.018387    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-068f74/auth-test-model-068f74-kserve-5874887fcf-n6cwg" podUID="a8b262d9-19c8-48d6-a9d0-2e5701cb7c34" containerName="main" containerID="cri-o://60188fb67ac08f1bed376ea5df1b4a80db12ce568d5db4373211dfd5f6fa7734" gracePeriod=45
Jul 07 12:06:52.263547 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:52.263521    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-068f74/auth-test-model-068f74-kserve-5874887fcf-n6cwg"
Jul 07 12:06:52.306898 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:52.306864    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5jvhk\" (UniqueName: \"kubernetes.io/projected/a8b262d9-19c8-48d6-a9d0-2e5701cb7c34-kube-api-access-5jvhk\") pod \"a8b262d9-19c8-48d6-a9d0-2e5701cb7c34\" (UID: \"a8b262d9-19c8-48d6-a9d0-2e5701cb7c34\") "
Jul 07 12:06:52.307119 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:52.306918    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8b262d9-19c8-48d6-a9d0-2e5701cb7c34-model-cache\") pod \"a8b262d9-19c8-48d6-a9d0-2e5701cb7c34\" (UID: \"a8b262d9-19c8-48d6-a9d0-2e5701cb7c34\") "
Jul 07 12:06:52.307119 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:52.307003    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8b262d9-19c8-48d6-a9d0-2e5701cb7c34-tls-certs\") pod \"a8b262d9-19c8-48d6-a9d0-2e5701cb7c34\" (UID: \"a8b262d9-19c8-48d6-a9d0-2e5701cb7c34\") "
Jul 07 12:06:52.307119 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:52.307047    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8b262d9-19c8-48d6-a9d0-2e5701cb7c34-dshm\") pod \"a8b262d9-19c8-48d6-a9d0-2e5701cb7c34\" (UID: \"a8b262d9-19c8-48d6-a9d0-2e5701cb7c34\") "
Jul 07 12:06:52.307119 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:52.307071    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8b262d9-19c8-48d6-a9d0-2e5701cb7c34-tmp-dir\") pod \"a8b262d9-19c8-48d6-a9d0-2e5701cb7c34\" (UID: \"a8b262d9-19c8-48d6-a9d0-2e5701cb7c34\") "
Jul 07 12:06:52.307119 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:52.307094    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8b262d9-19c8-48d6-a9d0-2e5701cb7c34-home\") pod \"a8b262d9-19c8-48d6-a9d0-2e5701cb7c34\" (UID: \"a8b262d9-19c8-48d6-a9d0-2e5701cb7c34\") "
Jul 07 12:06:52.307467 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:52.307254    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8b262d9-19c8-48d6-a9d0-2e5701cb7c34-model-cache" (OuterVolumeSpecName: "model-cache") pod "a8b262d9-19c8-48d6-a9d0-2e5701cb7c34" (UID: "a8b262d9-19c8-48d6-a9d0-2e5701cb7c34"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:06:52.307467 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:52.307414    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8b262d9-19c8-48d6-a9d0-2e5701cb7c34-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a8b262d9-19c8-48d6-a9d0-2e5701cb7c34" (UID: "a8b262d9-19c8-48d6-a9d0-2e5701cb7c34"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:06:52.307467 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:52.307439    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8b262d9-19c8-48d6-a9d0-2e5701cb7c34-model-cache\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:06:52.307467 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:52.307444    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8b262d9-19c8-48d6-a9d0-2e5701cb7c34-home" (OuterVolumeSpecName: "home") pod "a8b262d9-19c8-48d6-a9d0-2e5701cb7c34" (UID: "a8b262d9-19c8-48d6-a9d0-2e5701cb7c34"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:06:52.309228 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:52.309185    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8b262d9-19c8-48d6-a9d0-2e5701cb7c34-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a8b262d9-19c8-48d6-a9d0-2e5701cb7c34" (UID: "a8b262d9-19c8-48d6-a9d0-2e5701cb7c34"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 12:06:52.309345 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:52.309241    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8b262d9-19c8-48d6-a9d0-2e5701cb7c34-dshm" (OuterVolumeSpecName: "dshm") pod "a8b262d9-19c8-48d6-a9d0-2e5701cb7c34" (UID: "a8b262d9-19c8-48d6-a9d0-2e5701cb7c34"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:06:52.309345 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:52.309289    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8b262d9-19c8-48d6-a9d0-2e5701cb7c34-kube-api-access-5jvhk" (OuterVolumeSpecName: "kube-api-access-5jvhk") pod "a8b262d9-19c8-48d6-a9d0-2e5701cb7c34" (UID: "a8b262d9-19c8-48d6-a9d0-2e5701cb7c34"). InnerVolumeSpecName "kube-api-access-5jvhk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 12:06:52.408062 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:52.408018    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8b262d9-19c8-48d6-a9d0-2e5701cb7c34-tls-certs\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:06:52.408062 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:52.408055    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8b262d9-19c8-48d6-a9d0-2e5701cb7c34-dshm\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:06:52.408062 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:52.408065    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8b262d9-19c8-48d6-a9d0-2e5701cb7c34-tmp-dir\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:06:52.408062 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:52.408072    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8b262d9-19c8-48d6-a9d0-2e5701cb7c34-home\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:06:52.408382 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:52.408081    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5jvhk\" (UniqueName: \"kubernetes.io/projected/a8b262d9-19c8-48d6-a9d0-2e5701cb7c34-kube-api-access-5jvhk\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:06:52.604375 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:52.604271    2570 generic.go:356] "Generic (PLEG): container finished" podID="a8b262d9-19c8-48d6-a9d0-2e5701cb7c34" containerID="60188fb67ac08f1bed376ea5df1b4a80db12ce568d5db4373211dfd5f6fa7734" exitCode=0
Jul 07 12:06:52.604375 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:52.604319    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-068f74/auth-test-model-068f74-kserve-5874887fcf-n6cwg"
Jul 07 12:06:52.604375 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:52.604325    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-068f74/auth-test-model-068f74-kserve-5874887fcf-n6cwg" event={"ID":"a8b262d9-19c8-48d6-a9d0-2e5701cb7c34","Type":"ContainerDied","Data":"60188fb67ac08f1bed376ea5df1b4a80db12ce568d5db4373211dfd5f6fa7734"}
Jul 07 12:06:52.604670 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:52.604381    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-068f74/auth-test-model-068f74-kserve-5874887fcf-n6cwg" event={"ID":"a8b262d9-19c8-48d6-a9d0-2e5701cb7c34","Type":"ContainerDied","Data":"78d4881ed8b41833d438fa984d488c052b9b31569a29c82653290255a7b62265"}
Jul 07 12:06:52.604670 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:52.604403    2570 scope.go:117] "RemoveContainer" containerID="60188fb67ac08f1bed376ea5df1b4a80db12ce568d5db4373211dfd5f6fa7734"
Jul 07 12:06:52.614169 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:52.614146    2570 scope.go:117] "RemoveContainer" containerID="60188fb67ac08f1bed376ea5df1b4a80db12ce568d5db4373211dfd5f6fa7734"
Jul 07 12:06:52.614506 ip-10-0-135-78 kubenswrapper[2570]: E0707 12:06:52.614481    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60188fb67ac08f1bed376ea5df1b4a80db12ce568d5db4373211dfd5f6fa7734\": container with ID starting with 60188fb67ac08f1bed376ea5df1b4a80db12ce568d5db4373211dfd5f6fa7734 not found: ID does not exist" containerID="60188fb67ac08f1bed376ea5df1b4a80db12ce568d5db4373211dfd5f6fa7734"
Jul 07 12:06:52.614569 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:52.614517    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60188fb67ac08f1bed376ea5df1b4a80db12ce568d5db4373211dfd5f6fa7734"} err="failed to get container status \"60188fb67ac08f1bed376ea5df1b4a80db12ce568d5db4373211dfd5f6fa7734\": rpc error: code = NotFound desc = could not find container \"60188fb67ac08f1bed376ea5df1b4a80db12ce568d5db4373211dfd5f6fa7734\": container with ID starting with 60188fb67ac08f1bed376ea5df1b4a80db12ce568d5db4373211dfd5f6fa7734 not found: ID does not exist"
Jul 07 12:06:52.626696 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:52.626671    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-068f74/auth-test-model-068f74-kserve-5874887fcf-n6cwg"]
Jul 07 12:06:52.632528 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:52.632501    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-068f74/auth-test-model-068f74-kserve-5874887fcf-n6cwg"]
Jul 07 12:06:53.938286 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:53.938254    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8b262d9-19c8-48d6-a9d0-2e5701cb7c34" path="/var/lib/kubelet/pods/a8b262d9-19c8-48d6-a9d0-2e5701cb7c34/volumes"
Jul 07 12:06:54.069710 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:54.069679    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ktbpw_faf6f343-6515-4b2c-a34e-286cd2d9ec7a/console-operator/1.log"
Jul 07 12:06:54.080029 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:54.080007    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ktbpw_faf6f343-6515-4b2c-a34e-286cd2d9ec7a/console-operator/1.log"
Jul 07 12:06:56.728994 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:56.728956    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-10f898/sub-test-model-10f898-kserve-7cd676f44c-d9lqg"
Jul 07 12:06:56.729464 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:56.729005    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-10f898/sub-test-model-10f898-kserve-7cd676f44c-d9lqg"
Jul 07 12:06:56.741796 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:56.741770    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-10f898/sub-test-model-10f898-kserve-7cd676f44c-d9lqg"
Jul 07 12:06:57.642271 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:06:57.642241    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-10f898/sub-test-model-10f898-kserve-7cd676f44c-d9lqg"
Jul 07 12:07:06.727717 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:07:06.727678    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-068f74/sub-test-model-068f74-kserve-5d895dfbdf-77tsz"]
Jul 07 12:07:06.728163 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:07:06.728041    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8b262d9-19c8-48d6-a9d0-2e5701cb7c34" containerName="main"
Jul 07 12:07:06.728163 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:07:06.728053    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8b262d9-19c8-48d6-a9d0-2e5701cb7c34" containerName="main"
Jul 07 12:07:06.728163 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:07:06.728067    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fde728ff-967a-4629-b9be-5dd4bab42fcf" containerName="main"
Jul 07 12:07:06.728163 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:07:06.728072    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="fde728ff-967a-4629-b9be-5dd4bab42fcf" containerName="main"
Jul 07 12:07:06.728163 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:07:06.728145    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8b262d9-19c8-48d6-a9d0-2e5701cb7c34" containerName="main"
Jul 07 12:07:06.728163 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:07:06.728153    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="fde728ff-967a-4629-b9be-5dd4bab42fcf" containerName="main"
Jul 07 12:07:06.730240 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:07:06.730204    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-068f74/sub-test-model-068f74-kserve-5d895dfbdf-77tsz"
Jul 07 12:07:06.733152 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:07:06.733126    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-068f74\"/\"openshift-service-ca.crt\""
Jul 07 12:07:06.734325 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:07:06.734302    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-068f74\"/\"default-dockercfg-ttmbl\""
Jul 07 12:07:06.734456 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:07:06.734357    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-068f74\"/\"kube-root-ca.crt\""
Jul 07 12:07:06.734456 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:07:06.734306    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-068f74\"/\"sub-test-model-068f74-kserve-self-signed-certs\""
Jul 07 12:07:06.740496 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:07:06.740472    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-068f74/sub-test-model-068f74-kserve-5d895dfbdf-77tsz"]
Jul 07 12:07:06.848948 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:07:06.848905    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1225384b-1335-41f4-808d-63a766626ae8-dshm\") pod \"sub-test-model-068f74-kserve-5d895dfbdf-77tsz\" (UID: \"1225384b-1335-41f4-808d-63a766626ae8\") " pod="ai-tenant-e2e-shared-b-068f74/sub-test-model-068f74-kserve-5d895dfbdf-77tsz"
Jul 07 12:07:06.849107 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:07:06.848958    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1225384b-1335-41f4-808d-63a766626ae8-home\") pod \"sub-test-model-068f74-kserve-5d895dfbdf-77tsz\" (UID: \"1225384b-1335-41f4-808d-63a766626ae8\") " pod="ai-tenant-e2e-shared-b-068f74/sub-test-model-068f74-kserve-5d895dfbdf-77tsz"
Jul 07 12:07:06.849107 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:07:06.849037    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lp4n\" (UniqueName: \"kubernetes.io/projected/1225384b-1335-41f4-808d-63a766626ae8-kube-api-access-9lp4n\") pod \"sub-test-model-068f74-kserve-5d895dfbdf-77tsz\" (UID: \"1225384b-1335-41f4-808d-63a766626ae8\") " pod="ai-tenant-e2e-shared-b-068f74/sub-test-model-068f74-kserve-5d895dfbdf-77tsz"
Jul 07 12:07:06.849107 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:07:06.849095    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1225384b-1335-41f4-808d-63a766626ae8-tmp-dir\") pod \"sub-test-model-068f74-kserve-5d895dfbdf-77tsz\" (UID: \"1225384b-1335-41f4-808d-63a766626ae8\") " pod="ai-tenant-e2e-shared-b-068f74/sub-test-model-068f74-kserve-5d895dfbdf-77tsz"
Jul 07 12:07:06.849273 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:07:06.849166    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1225384b-1335-41f4-808d-63a766626ae8-tls-certs\") pod \"sub-test-model-068f74-kserve-5d895dfbdf-77tsz\" (UID: \"1225384b-1335-41f4-808d-63a766626ae8\") " pod="ai-tenant-e2e-shared-b-068f74/sub-test-model-068f74-kserve-5d895dfbdf-77tsz"
Jul 07 12:07:06.849273 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:07:06.849197    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1225384b-1335-41f4-808d-63a766626ae8-model-cache\") pod \"sub-test-model-068f74-kserve-5d895dfbdf-77tsz\" (UID: \"1225384b-1335-41f4-808d-63a766626ae8\") " pod="ai-tenant-e2e-shared-b-068f74/sub-test-model-068f74-kserve-5d895dfbdf-77tsz"
Jul 07 12:07:06.950396 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:07:06.950364    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1225384b-1335-41f4-808d-63a766626ae8-tmp-dir\") pod \"sub-test-model-068f74-kserve-5d895dfbdf-77tsz\" (UID: \"1225384b-1335-41f4-808d-63a766626ae8\") " pod="ai-tenant-e2e-shared-b-068f74/sub-test-model-068f74-kserve-5d895dfbdf-77tsz"
Jul 07 12:07:06.950697 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:07:06.950425    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1225384b-1335-41f4-808d-63a766626ae8-tls-certs\") pod \"sub-test-model-068f74-kserve-5d895dfbdf-77tsz\" (UID: \"1225384b-1335-41f4-808d-63a766626ae8\") " pod="ai-tenant-e2e-shared-b-068f74/sub-test-model-068f74-kserve-5d895dfbdf-77tsz"
Jul 07 12:07:06.950697 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:07:06.950453    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1225384b-1335-41f4-808d-63a766626ae8-model-cache\") pod \"sub-test-model-068f74-kserve-5d895dfbdf-77tsz\" (UID: \"1225384b-1335-41f4-808d-63a766626ae8\") " pod="ai-tenant-e2e-shared-b-068f74/sub-test-model-068f74-kserve-5d895dfbdf-77tsz"
Jul 07 12:07:06.950697 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:07:06.950480    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1225384b-1335-41f4-808d-63a766626ae8-dshm\") pod \"sub-test-model-068f74-kserve-5d895dfbdf-77tsz\" (UID: \"1225384b-1335-41f4-808d-63a766626ae8\") " pod="ai-tenant-e2e-shared-b-068f74/sub-test-model-068f74-kserve-5d895dfbdf-77tsz"
Jul 07 12:07:06.950697 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:07:06.950510    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1225384b-1335-41f4-808d-63a766626ae8-home\") pod \"sub-test-model-068f74-kserve-5d895dfbdf-77tsz\" (UID: \"1225384b-1335-41f4-808d-63a766626ae8\") " pod="ai-tenant-e2e-shared-b-068f74/sub-test-model-068f74-kserve-5d895dfbdf-77tsz"
Jul 07 12:07:06.950697 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:07:06.950563    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lp4n\" (UniqueName: \"kubernetes.io/projected/1225384b-1335-41f4-808d-63a766626ae8-kube-api-access-9lp4n\") pod \"sub-test-model-068f74-kserve-5d895dfbdf-77tsz\" (UID: \"1225384b-1335-41f4-808d-63a766626ae8\") " pod="ai-tenant-e2e-shared-b-068f74/sub-test-model-068f74-kserve-5d895dfbdf-77tsz"
Jul 07 12:07:06.951022 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:07:06.950735    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1225384b-1335-41f4-808d-63a766626ae8-tmp-dir\") pod \"sub-test-model-068f74-kserve-5d895dfbdf-77tsz\" (UID: \"1225384b-1335-41f4-808d-63a766626ae8\") " pod="ai-tenant-e2e-shared-b-068f74/sub-test-model-068f74-kserve-5d895dfbdf-77tsz"
Jul 07 12:07:06.951022 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:07:06.950803    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1225384b-1335-41f4-808d-63a766626ae8-model-cache\") pod \"sub-test-model-068f74-kserve-5d895dfbdf-77tsz\" (UID: \"1225384b-1335-41f4-808d-63a766626ae8\") " pod="ai-tenant-e2e-shared-b-068f74/sub-test-model-068f74-kserve-5d895dfbdf-77tsz"
Jul 07 12:07:06.951022 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:07:06.950842    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1225384b-1335-41f4-808d-63a766626ae8-home\") pod \"sub-test-model-068f74-kserve-5d895dfbdf-77tsz\" (UID: \"1225384b-1335-41f4-808d-63a766626ae8\") " pod="ai-tenant-e2e-shared-b-068f74/sub-test-model-068f74-kserve-5d895dfbdf-77tsz"
Jul 07 12:07:06.952775 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:07:06.952752    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1225384b-1335-41f4-808d-63a766626ae8-dshm\") pod \"sub-test-model-068f74-kserve-5d895dfbdf-77tsz\" (UID: \"1225384b-1335-41f4-808d-63a766626ae8\") " pod="ai-tenant-e2e-shared-b-068f74/sub-test-model-068f74-kserve-5d895dfbdf-77tsz"
Jul 07 12:07:06.953031 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:07:06.953013    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1225384b-1335-41f4-808d-63a766626ae8-tls-certs\") pod \"sub-test-model-068f74-kserve-5d895dfbdf-77tsz\" (UID: \"1225384b-1335-41f4-808d-63a766626ae8\") " pod="ai-tenant-e2e-shared-b-068f74/sub-test-model-068f74-kserve-5d895dfbdf-77tsz"
Jul 07 12:07:06.959809 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:07:06.959780    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lp4n\" (UniqueName: \"kubernetes.io/projected/1225384b-1335-41f4-808d-63a766626ae8-kube-api-access-9lp4n\") pod \"sub-test-model-068f74-kserve-5d895dfbdf-77tsz\" (UID: \"1225384b-1335-41f4-808d-63a766626ae8\") " pod="ai-tenant-e2e-shared-b-068f74/sub-test-model-068f74-kserve-5d895dfbdf-77tsz"
Jul 07 12:07:07.041428 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:07:07.041333    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-068f74/sub-test-model-068f74-kserve-5d895dfbdf-77tsz"
Jul 07 12:07:07.179880 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:07:07.179849    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-068f74/sub-test-model-068f74-kserve-5d895dfbdf-77tsz"]
Jul 07 12:07:07.180988 ip-10-0-135-78 kubenswrapper[2570]: W0707 12:07:07.180959    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1225384b_1335_41f4_808d_63a766626ae8.slice/crio-97a2afa879c1d1332c900ac7bdc49b589082c471cf231eb93ac2d1842ebaa925 WatchSource:0}: Error finding container 97a2afa879c1d1332c900ac7bdc49b589082c471cf231eb93ac2d1842ebaa925: Status 404 returned error can't find the container with id 97a2afa879c1d1332c900ac7bdc49b589082c471cf231eb93ac2d1842ebaa925
Jul 07 12:07:07.683767 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:07:07.683723    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-068f74/sub-test-model-068f74-kserve-5d895dfbdf-77tsz" event={"ID":"1225384b-1335-41f4-808d-63a766626ae8","Type":"ContainerStarted","Data":"f649e87e4770ada5e98061ad902c35b13d0d7b96fa15170ad4e06ce59e14786e"}
Jul 07 12:07:07.683767 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:07:07.683772    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-068f74/sub-test-model-068f74-kserve-5d895dfbdf-77tsz" event={"ID":"1225384b-1335-41f4-808d-63a766626ae8","Type":"ContainerStarted","Data":"97a2afa879c1d1332c900ac7bdc49b589082c471cf231eb93ac2d1842ebaa925"}
Jul 07 12:07:07.704260 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:07:07.704196    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-068f74/sub-test-model-068f74-kserve-5d895dfbdf-77tsz" podStartSLOduration=1.704181818 podStartE2EDuration="1.704181818s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:07:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:07:07.703053982 +0000 UTC m=+2415.676694393" watchObservedRunningTime="2026-07-07 12:07:07.704181818 +0000 UTC m=+2415.677822228"
Jul 07 12:07:17.041841 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:07:17.041736    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-068f74/sub-test-model-068f74-kserve-5d895dfbdf-77tsz"
Jul 07 12:07:17.041841 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:07:17.041812    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-068f74/sub-test-model-068f74-kserve-5d895dfbdf-77tsz"
Jul 07 12:07:17.054304 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:07:17.054278    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-068f74/sub-test-model-068f74-kserve-5d895dfbdf-77tsz"
Jul 07 12:07:17.747366 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:07:17.747335    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-068f74/sub-test-model-068f74-kserve-5d895dfbdf-77tsz"
Jul 07 12:07:53.908349 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:07:53.908312    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-10f898/sub-test-model-10f898-kserve-7cd676f44c-d9lqg"]
Jul 07 12:07:54.941142 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:07:54.941102    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-068f74/sub-test-model-068f74-kserve-5d895dfbdf-77tsz"]
Jul 07 12:08:08.930723 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:08.930681    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-10f898/sub-test-model-10f898-kserve-7cd676f44c-d9lqg" podUID="bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a" containerName="main" containerID="cri-o://27b1343e5865161f684b8de28386ec9e8434f5214fe056ec8f9f05ef0d36df4e" gracePeriod=45
Jul 07 12:08:09.184766 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:09.184739    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-10f898/sub-test-model-10f898-kserve-7cd676f44c-d9lqg"
Jul 07 12:08:09.259451 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:09.259403    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a-model-cache\") pod \"bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a\" (UID: \"bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a\") "
Jul 07 12:08:09.259653 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:09.259487    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f4ksn\" (UniqueName: \"kubernetes.io/projected/bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a-kube-api-access-f4ksn\") pod \"bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a\" (UID: \"bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a\") "
Jul 07 12:08:09.259653 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:09.259530    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a-tls-certs\") pod \"bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a\" (UID: \"bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a\") "
Jul 07 12:08:09.259653 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:09.259572    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a-tmp-dir\") pod \"bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a\" (UID: \"bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a\") "
Jul 07 12:08:09.259852 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:09.259660    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a-model-cache" (OuterVolumeSpecName: "model-cache") pod "bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a" (UID: "bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:08:09.259852 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:09.259691    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a-home\") pod \"bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a\" (UID: \"bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a\") "
Jul 07 12:08:09.259852 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:09.259785    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a-dshm\") pod \"bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a\" (UID: \"bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a\") "
Jul 07 12:08:09.259852 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:09.259808    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a" (UID: "bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:08:09.260018 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:09.259969    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a-home" (OuterVolumeSpecName: "home") pod "bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a" (UID: "bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:08:09.260160 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:09.260138    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a-model-cache\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:08:09.260268 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:09.260167    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a-tmp-dir\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:08:09.260268 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:09.260182    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a-home\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:08:09.261763 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:09.261730    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a-dshm" (OuterVolumeSpecName: "dshm") pod "bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a" (UID: "bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:08:09.261888 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:09.261774    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a" (UID: "bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 12:08:09.261888 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:09.261825    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a-kube-api-access-f4ksn" (OuterVolumeSpecName: "kube-api-access-f4ksn") pod "bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a" (UID: "bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a"). InnerVolumeSpecName "kube-api-access-f4ksn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 12:08:09.361665 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:09.361608    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f4ksn\" (UniqueName: \"kubernetes.io/projected/bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a-kube-api-access-f4ksn\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:08:09.361665 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:09.361657    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a-tls-certs\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:08:09.361665 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:09.361667    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a-dshm\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:08:09.618379 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:09.618293    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-10f898/rate-test-model-10f898-kserve-648f468475-49ppf"]
Jul 07 12:08:09.618687 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:09.618674    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a" containerName="main"
Jul 07 12:08:09.618733 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:09.618689    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a" containerName="main"
Jul 07 12:08:09.618780 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:09.618755    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a" containerName="main"
Jul 07 12:08:09.620768 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:09.620749    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-10f898/rate-test-model-10f898-kserve-648f468475-49ppf"
Jul 07 12:08:09.623507 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:09.623486    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-10f898\"/\"rate-test-model-10f898-kserve-self-signed-certs\""
Jul 07 12:08:09.631049 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:09.631024    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-10f898/rate-test-model-10f898-kserve-648f468475-49ppf"]
Jul 07 12:08:09.665036 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:09.664994    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/250f3d9c-4146-404e-80dc-caa57983e053-home\") pod \"rate-test-model-10f898-kserve-648f468475-49ppf\" (UID: \"250f3d9c-4146-404e-80dc-caa57983e053\") " pod="ai-tenant-e2e-shared-a-10f898/rate-test-model-10f898-kserve-648f468475-49ppf"
Jul 07 12:08:09.665036 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:09.665041    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/250f3d9c-4146-404e-80dc-caa57983e053-dshm\") pod \"rate-test-model-10f898-kserve-648f468475-49ppf\" (UID: \"250f3d9c-4146-404e-80dc-caa57983e053\") " pod="ai-tenant-e2e-shared-a-10f898/rate-test-model-10f898-kserve-648f468475-49ppf"
Jul 07 12:08:09.665333 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:09.665132    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgpn9\" (UniqueName: \"kubernetes.io/projected/250f3d9c-4146-404e-80dc-caa57983e053-kube-api-access-xgpn9\") pod \"rate-test-model-10f898-kserve-648f468475-49ppf\" (UID: \"250f3d9c-4146-404e-80dc-caa57983e053\") " pod="ai-tenant-e2e-shared-a-10f898/rate-test-model-10f898-kserve-648f468475-49ppf"
Jul 07 12:08:09.665333 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:09.665188    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/250f3d9c-4146-404e-80dc-caa57983e053-model-cache\") pod \"rate-test-model-10f898-kserve-648f468475-49ppf\" (UID: \"250f3d9c-4146-404e-80dc-caa57983e053\") " pod="ai-tenant-e2e-shared-a-10f898/rate-test-model-10f898-kserve-648f468475-49ppf"
Jul 07 12:08:09.665333 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:09.665292    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/250f3d9c-4146-404e-80dc-caa57983e053-tmp-dir\") pod \"rate-test-model-10f898-kserve-648f468475-49ppf\" (UID: \"250f3d9c-4146-404e-80dc-caa57983e053\") " pod="ai-tenant-e2e-shared-a-10f898/rate-test-model-10f898-kserve-648f468475-49ppf"
Jul 07 12:08:09.665333 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:09.665328    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/250f3d9c-4146-404e-80dc-caa57983e053-tls-certs\") pod \"rate-test-model-10f898-kserve-648f468475-49ppf\" (UID: \"250f3d9c-4146-404e-80dc-caa57983e053\") " pod="ai-tenant-e2e-shared-a-10f898/rate-test-model-10f898-kserve-648f468475-49ppf"
Jul 07 12:08:09.766862 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:09.766813    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/250f3d9c-4146-404e-80dc-caa57983e053-home\") pod \"rate-test-model-10f898-kserve-648f468475-49ppf\" (UID: \"250f3d9c-4146-404e-80dc-caa57983e053\") " pod="ai-tenant-e2e-shared-a-10f898/rate-test-model-10f898-kserve-648f468475-49ppf"
Jul 07 12:08:09.766862 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:09.766861    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/250f3d9c-4146-404e-80dc-caa57983e053-dshm\") pod \"rate-test-model-10f898-kserve-648f468475-49ppf\" (UID: \"250f3d9c-4146-404e-80dc-caa57983e053\") " pod="ai-tenant-e2e-shared-a-10f898/rate-test-model-10f898-kserve-648f468475-49ppf"
Jul 07 12:08:09.767139 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:09.766904    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgpn9\" (UniqueName: \"kubernetes.io/projected/250f3d9c-4146-404e-80dc-caa57983e053-kube-api-access-xgpn9\") pod \"rate-test-model-10f898-kserve-648f468475-49ppf\" (UID: \"250f3d9c-4146-404e-80dc-caa57983e053\") " pod="ai-tenant-e2e-shared-a-10f898/rate-test-model-10f898-kserve-648f468475-49ppf"
Jul 07 12:08:09.767139 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:09.766937    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/250f3d9c-4146-404e-80dc-caa57983e053-model-cache\") pod \"rate-test-model-10f898-kserve-648f468475-49ppf\" (UID: \"250f3d9c-4146-404e-80dc-caa57983e053\") " pod="ai-tenant-e2e-shared-a-10f898/rate-test-model-10f898-kserve-648f468475-49ppf"
Jul 07 12:08:09.767139 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:09.766963    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/250f3d9c-4146-404e-80dc-caa57983e053-tmp-dir\") pod \"rate-test-model-10f898-kserve-648f468475-49ppf\" (UID: \"250f3d9c-4146-404e-80dc-caa57983e053\") " pod="ai-tenant-e2e-shared-a-10f898/rate-test-model-10f898-kserve-648f468475-49ppf"
Jul 07 12:08:09.767139 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:09.766985    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/250f3d9c-4146-404e-80dc-caa57983e053-tls-certs\") pod \"rate-test-model-10f898-kserve-648f468475-49ppf\" (UID: \"250f3d9c-4146-404e-80dc-caa57983e053\") " pod="ai-tenant-e2e-shared-a-10f898/rate-test-model-10f898-kserve-648f468475-49ppf"
Jul 07 12:08:09.767414 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:09.767315    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/250f3d9c-4146-404e-80dc-caa57983e053-home\") pod \"rate-test-model-10f898-kserve-648f468475-49ppf\" (UID: \"250f3d9c-4146-404e-80dc-caa57983e053\") " pod="ai-tenant-e2e-shared-a-10f898/rate-test-model-10f898-kserve-648f468475-49ppf"
Jul 07 12:08:09.767414 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:09.767368    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/250f3d9c-4146-404e-80dc-caa57983e053-model-cache\") pod \"rate-test-model-10f898-kserve-648f468475-49ppf\" (UID: \"250f3d9c-4146-404e-80dc-caa57983e053\") " pod="ai-tenant-e2e-shared-a-10f898/rate-test-model-10f898-kserve-648f468475-49ppf"
Jul 07 12:08:09.767487 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:09.767420    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/250f3d9c-4146-404e-80dc-caa57983e053-tmp-dir\") pod \"rate-test-model-10f898-kserve-648f468475-49ppf\" (UID: \"250f3d9c-4146-404e-80dc-caa57983e053\") " pod="ai-tenant-e2e-shared-a-10f898/rate-test-model-10f898-kserve-648f468475-49ppf"
Jul 07 12:08:09.769286 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:09.769268    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/250f3d9c-4146-404e-80dc-caa57983e053-dshm\") pod \"rate-test-model-10f898-kserve-648f468475-49ppf\" (UID: \"250f3d9c-4146-404e-80dc-caa57983e053\") " pod="ai-tenant-e2e-shared-a-10f898/rate-test-model-10f898-kserve-648f468475-49ppf"
Jul 07 12:08:09.769468 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:09.769452    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/250f3d9c-4146-404e-80dc-caa57983e053-tls-certs\") pod \"rate-test-model-10f898-kserve-648f468475-49ppf\" (UID: \"250f3d9c-4146-404e-80dc-caa57983e053\") " pod="ai-tenant-e2e-shared-a-10f898/rate-test-model-10f898-kserve-648f468475-49ppf"
Jul 07 12:08:09.775221 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:09.775192    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgpn9\" (UniqueName: \"kubernetes.io/projected/250f3d9c-4146-404e-80dc-caa57983e053-kube-api-access-xgpn9\") pod \"rate-test-model-10f898-kserve-648f468475-49ppf\" (UID: \"250f3d9c-4146-404e-80dc-caa57983e053\") " pod="ai-tenant-e2e-shared-a-10f898/rate-test-model-10f898-kserve-648f468475-49ppf"
Jul 07 12:08:09.932711 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:09.932673    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-10f898/rate-test-model-10f898-kserve-648f468475-49ppf"
Jul 07 12:08:09.961688 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:09.961651    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-068f74/sub-test-model-068f74-kserve-5d895dfbdf-77tsz" podUID="1225384b-1335-41f4-808d-63a766626ae8" containerName="main" containerID="cri-o://f649e87e4770ada5e98061ad902c35b13d0d7b96fa15170ad4e06ce59e14786e" gracePeriod=45
Jul 07 12:08:10.031004 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:10.030962    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-10f898/sub-test-model-10f898-kserve-7cd676f44c-d9lqg" event={"ID":"bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a","Type":"ContainerDied","Data":"27b1343e5865161f684b8de28386ec9e8434f5214fe056ec8f9f05ef0d36df4e"}
Jul 07 12:08:10.031198 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:10.031024    2570 scope.go:117] "RemoveContainer" containerID="27b1343e5865161f684b8de28386ec9e8434f5214fe056ec8f9f05ef0d36df4e"
Jul 07 12:08:10.031787 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:10.030832    2570 generic.go:356] "Generic (PLEG): container finished" podID="bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a" containerID="27b1343e5865161f684b8de28386ec9e8434f5214fe056ec8f9f05ef0d36df4e" exitCode=0
Jul 07 12:08:10.031787 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:10.031520    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-10f898/sub-test-model-10f898-kserve-7cd676f44c-d9lqg"
Jul 07 12:08:10.031787 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:10.031621    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-10f898/sub-test-model-10f898-kserve-7cd676f44c-d9lqg" event={"ID":"bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a","Type":"ContainerDied","Data":"ade36546bedef0b719aca04616c7f84e25733bcb178ce503aa5b680bd495bd4d"}
Jul 07 12:08:10.054634 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:10.054526    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-10f898/sub-test-model-10f898-kserve-7cd676f44c-d9lqg"]
Jul 07 12:08:10.058390 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:10.057520    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-10f898/sub-test-model-10f898-kserve-7cd676f44c-d9lqg"]
Jul 07 12:08:10.077903 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:10.077875    2570 scope.go:117] "RemoveContainer" containerID="27b1343e5865161f684b8de28386ec9e8434f5214fe056ec8f9f05ef0d36df4e"
Jul 07 12:08:10.078401 ip-10-0-135-78 kubenswrapper[2570]: E0707 12:08:10.078373    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27b1343e5865161f684b8de28386ec9e8434f5214fe056ec8f9f05ef0d36df4e\": container with ID starting with 27b1343e5865161f684b8de28386ec9e8434f5214fe056ec8f9f05ef0d36df4e not found: ID does not exist" containerID="27b1343e5865161f684b8de28386ec9e8434f5214fe056ec8f9f05ef0d36df4e"
Jul 07 12:08:10.078468 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:10.078421    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27b1343e5865161f684b8de28386ec9e8434f5214fe056ec8f9f05ef0d36df4e"} err="failed to get container status \"27b1343e5865161f684b8de28386ec9e8434f5214fe056ec8f9f05ef0d36df4e\": rpc error: code = NotFound desc = could not find container \"27b1343e5865161f684b8de28386ec9e8434f5214fe056ec8f9f05ef0d36df4e\": container with ID starting with 27b1343e5865161f684b8de28386ec9e8434f5214fe056ec8f9f05ef0d36df4e not found: ID does not exist"
Jul 07 12:08:10.093764 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:10.093714    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-10f898/rate-test-model-10f898-kserve-648f468475-49ppf"]
Jul 07 12:08:10.097142 ip-10-0-135-78 kubenswrapper[2570]: W0707 12:08:10.097107    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod250f3d9c_4146_404e_80dc_caa57983e053.slice/crio-ef8e0ce0a7a32b3fcf1f1fb2e304b90c8ea43941cd8f26de1577fe2544616b44 WatchSource:0}: Error finding container ef8e0ce0a7a32b3fcf1f1fb2e304b90c8ea43941cd8f26de1577fe2544616b44: Status 404 returned error can't find the container with id ef8e0ce0a7a32b3fcf1f1fb2e304b90c8ea43941cd8f26de1577fe2544616b44
Jul 07 12:08:10.228259 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:10.228232    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-068f74/sub-test-model-068f74-kserve-5d895dfbdf-77tsz"
Jul 07 12:08:10.272712 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:10.272673    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1225384b-1335-41f4-808d-63a766626ae8-tls-certs\") pod \"1225384b-1335-41f4-808d-63a766626ae8\" (UID: \"1225384b-1335-41f4-808d-63a766626ae8\") "
Jul 07 12:08:10.272928 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:10.272771    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9lp4n\" (UniqueName: \"kubernetes.io/projected/1225384b-1335-41f4-808d-63a766626ae8-kube-api-access-9lp4n\") pod \"1225384b-1335-41f4-808d-63a766626ae8\" (UID: \"1225384b-1335-41f4-808d-63a766626ae8\") "
Jul 07 12:08:10.273000 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:10.272929    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1225384b-1335-41f4-808d-63a766626ae8-tmp-dir\") pod \"1225384b-1335-41f4-808d-63a766626ae8\" (UID: \"1225384b-1335-41f4-808d-63a766626ae8\") "
Jul 07 12:08:10.273103 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:10.273077    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1225384b-1335-41f4-808d-63a766626ae8-home\") pod \"1225384b-1335-41f4-808d-63a766626ae8\" (UID: \"1225384b-1335-41f4-808d-63a766626ae8\") "
Jul 07 12:08:10.273179 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:10.273119    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1225384b-1335-41f4-808d-63a766626ae8-model-cache\") pod \"1225384b-1335-41f4-808d-63a766626ae8\" (UID: \"1225384b-1335-41f4-808d-63a766626ae8\") "
Jul 07 12:08:10.273179 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:10.273152    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1225384b-1335-41f4-808d-63a766626ae8-dshm\") pod \"1225384b-1335-41f4-808d-63a766626ae8\" (UID: \"1225384b-1335-41f4-808d-63a766626ae8\") "
Jul 07 12:08:10.273179 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:10.273157    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1225384b-1335-41f4-808d-63a766626ae8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1225384b-1335-41f4-808d-63a766626ae8" (UID: "1225384b-1335-41f4-808d-63a766626ae8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:08:10.273424 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:10.273341    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1225384b-1335-41f4-808d-63a766626ae8-home" (OuterVolumeSpecName: "home") pod "1225384b-1335-41f4-808d-63a766626ae8" (UID: "1225384b-1335-41f4-808d-63a766626ae8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:08:10.273643 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:10.273571    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1225384b-1335-41f4-808d-63a766626ae8-tmp-dir\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:08:10.273643 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:10.273593    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1225384b-1335-41f4-808d-63a766626ae8-home\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:08:10.273825 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:10.273716    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1225384b-1335-41f4-808d-63a766626ae8-model-cache" (OuterVolumeSpecName: "model-cache") pod "1225384b-1335-41f4-808d-63a766626ae8" (UID: "1225384b-1335-41f4-808d-63a766626ae8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:08:10.275466 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:10.275400    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1225384b-1335-41f4-808d-63a766626ae8-dshm" (OuterVolumeSpecName: "dshm") pod "1225384b-1335-41f4-808d-63a766626ae8" (UID: "1225384b-1335-41f4-808d-63a766626ae8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:08:10.275466 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:10.275403    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1225384b-1335-41f4-808d-63a766626ae8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1225384b-1335-41f4-808d-63a766626ae8" (UID: "1225384b-1335-41f4-808d-63a766626ae8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 12:08:10.275466 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:10.275449    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1225384b-1335-41f4-808d-63a766626ae8-kube-api-access-9lp4n" (OuterVolumeSpecName: "kube-api-access-9lp4n") pod "1225384b-1335-41f4-808d-63a766626ae8" (UID: "1225384b-1335-41f4-808d-63a766626ae8"). InnerVolumeSpecName "kube-api-access-9lp4n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 12:08:10.374575 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:10.374538    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9lp4n\" (UniqueName: \"kubernetes.io/projected/1225384b-1335-41f4-808d-63a766626ae8-kube-api-access-9lp4n\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:08:10.374575 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:10.374569    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1225384b-1335-41f4-808d-63a766626ae8-model-cache\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:08:10.374575 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:10.374580    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1225384b-1335-41f4-808d-63a766626ae8-dshm\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:08:10.374833 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:10.374589    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1225384b-1335-41f4-808d-63a766626ae8-tls-certs\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:08:11.039553 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:11.039512    2570 generic.go:356] "Generic (PLEG): container finished" podID="1225384b-1335-41f4-808d-63a766626ae8" containerID="f649e87e4770ada5e98061ad902c35b13d0d7b96fa15170ad4e06ce59e14786e" exitCode=0
Jul 07 12:08:11.040068 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:11.039583    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-068f74/sub-test-model-068f74-kserve-5d895dfbdf-77tsz"
Jul 07 12:08:11.040068 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:11.039603    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-068f74/sub-test-model-068f74-kserve-5d895dfbdf-77tsz" event={"ID":"1225384b-1335-41f4-808d-63a766626ae8","Type":"ContainerDied","Data":"f649e87e4770ada5e98061ad902c35b13d0d7b96fa15170ad4e06ce59e14786e"}
Jul 07 12:08:11.040068 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:11.039652    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-068f74/sub-test-model-068f74-kserve-5d895dfbdf-77tsz" event={"ID":"1225384b-1335-41f4-808d-63a766626ae8","Type":"ContainerDied","Data":"97a2afa879c1d1332c900ac7bdc49b589082c471cf231eb93ac2d1842ebaa925"}
Jul 07 12:08:11.040068 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:11.039675    2570 scope.go:117] "RemoveContainer" containerID="f649e87e4770ada5e98061ad902c35b13d0d7b96fa15170ad4e06ce59e14786e"
Jul 07 12:08:11.043329 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:11.043296    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-10f898/rate-test-model-10f898-kserve-648f468475-49ppf" event={"ID":"250f3d9c-4146-404e-80dc-caa57983e053","Type":"ContainerStarted","Data":"aa08632ec69cac674821c058229779d28e05ba5d77cbd3a1645bd9941c21e97c"}
Jul 07 12:08:11.043481 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:11.043347    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-10f898/rate-test-model-10f898-kserve-648f468475-49ppf" event={"ID":"250f3d9c-4146-404e-80dc-caa57983e053","Type":"ContainerStarted","Data":"ef8e0ce0a7a32b3fcf1f1fb2e304b90c8ea43941cd8f26de1577fe2544616b44"}
Jul 07 12:08:11.051963 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:11.051943    2570 scope.go:117] "RemoveContainer" containerID="f649e87e4770ada5e98061ad902c35b13d0d7b96fa15170ad4e06ce59e14786e"
Jul 07 12:08:11.052499 ip-10-0-135-78 kubenswrapper[2570]: E0707 12:08:11.052471    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f649e87e4770ada5e98061ad902c35b13d0d7b96fa15170ad4e06ce59e14786e\": container with ID starting with f649e87e4770ada5e98061ad902c35b13d0d7b96fa15170ad4e06ce59e14786e not found: ID does not exist" containerID="f649e87e4770ada5e98061ad902c35b13d0d7b96fa15170ad4e06ce59e14786e"
Jul 07 12:08:11.052612 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:11.052511    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f649e87e4770ada5e98061ad902c35b13d0d7b96fa15170ad4e06ce59e14786e"} err="failed to get container status \"f649e87e4770ada5e98061ad902c35b13d0d7b96fa15170ad4e06ce59e14786e\": rpc error: code = NotFound desc = could not find container \"f649e87e4770ada5e98061ad902c35b13d0d7b96fa15170ad4e06ce59e14786e\": container with ID starting with f649e87e4770ada5e98061ad902c35b13d0d7b96fa15170ad4e06ce59e14786e not found: ID does not exist"
Jul 07 12:08:11.061496 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:11.061439    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-10f898/rate-test-model-10f898-kserve-648f468475-49ppf" podStartSLOduration=2.061421346 podStartE2EDuration="2.061421346s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:08:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:08:11.059835427 +0000 UTC m=+2479.033475829" watchObservedRunningTime="2026-07-07 12:08:11.061421346 +0000 UTC m=+2479.035061802"
Jul 07 12:08:11.075933 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:11.075872    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-068f74/sub-test-model-068f74-kserve-5d895dfbdf-77tsz"]
Jul 07 12:08:11.080138 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:11.080102    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-068f74/sub-test-model-068f74-kserve-5d895dfbdf-77tsz"]
Jul 07 12:08:11.937931 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:11.937897    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1225384b-1335-41f4-808d-63a766626ae8" path="/var/lib/kubelet/pods/1225384b-1335-41f4-808d-63a766626ae8/volumes"
Jul 07 12:08:11.938341 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:11.938327    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a" path="/var/lib/kubelet/pods/bdc1b5dc-b11c-47f8-a7e1-95c5d385df2a/volumes"
Jul 07 12:08:19.939043 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:19.939009    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-10f898/rate-test-model-10f898-kserve-648f468475-49ppf"
Jul 07 12:08:19.939679 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:19.939057    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-10f898/rate-test-model-10f898-kserve-648f468475-49ppf"
Jul 07 12:08:19.946675 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:19.946648    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-10f898/rate-test-model-10f898-kserve-648f468475-49ppf"
Jul 07 12:08:20.110031 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:20.110001    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-10f898/rate-test-model-10f898-kserve-648f468475-49ppf"
Jul 07 12:08:44.526854 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:44.526732    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-068f74/rate-test-model-068f74-kserve-d848dd5c9-sjv95"]
Jul 07 12:08:44.527404 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:44.527381    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1225384b-1335-41f4-808d-63a766626ae8" containerName="main"
Jul 07 12:08:44.527458 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:44.527408    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="1225384b-1335-41f4-808d-63a766626ae8" containerName="main"
Jul 07 12:08:44.527544 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:44.527531    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="1225384b-1335-41f4-808d-63a766626ae8" containerName="main"
Jul 07 12:08:44.531091 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:44.531070    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-068f74/rate-test-model-068f74-kserve-d848dd5c9-sjv95"
Jul 07 12:08:44.534094 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:44.534071    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-068f74\"/\"kube-root-ca.crt\""
Jul 07 12:08:44.535406 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:44.535386    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-068f74\"/\"default-dockercfg-ttmbl\""
Jul 07 12:08:44.535554 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:44.535452    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-068f74\"/\"rate-test-model-068f74-kserve-self-signed-certs\""
Jul 07 12:08:44.535554 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:44.535454    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-068f74\"/\"openshift-service-ca.crt\""
Jul 07 12:08:44.539394 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:44.539366    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-068f74/rate-test-model-068f74-kserve-d848dd5c9-sjv95"]
Jul 07 12:08:44.608172 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:44.608144    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/222efa62-cd81-463f-b18d-474214fd2c71-tmp-dir\") pod \"rate-test-model-068f74-kserve-d848dd5c9-sjv95\" (UID: \"222efa62-cd81-463f-b18d-474214fd2c71\") " pod="ai-tenant-e2e-shared-b-068f74/rate-test-model-068f74-kserve-d848dd5c9-sjv95"
Jul 07 12:08:44.608388 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:44.608179    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g28wl\" (UniqueName: \"kubernetes.io/projected/222efa62-cd81-463f-b18d-474214fd2c71-kube-api-access-g28wl\") pod \"rate-test-model-068f74-kserve-d848dd5c9-sjv95\" (UID: \"222efa62-cd81-463f-b18d-474214fd2c71\") " pod="ai-tenant-e2e-shared-b-068f74/rate-test-model-068f74-kserve-d848dd5c9-sjv95"
Jul 07 12:08:44.608388 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:44.608266    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/222efa62-cd81-463f-b18d-474214fd2c71-tls-certs\") pod \"rate-test-model-068f74-kserve-d848dd5c9-sjv95\" (UID: \"222efa62-cd81-463f-b18d-474214fd2c71\") " pod="ai-tenant-e2e-shared-b-068f74/rate-test-model-068f74-kserve-d848dd5c9-sjv95"
Jul 07 12:08:44.608388 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:44.608286    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/222efa62-cd81-463f-b18d-474214fd2c71-dshm\") pod \"rate-test-model-068f74-kserve-d848dd5c9-sjv95\" (UID: \"222efa62-cd81-463f-b18d-474214fd2c71\") " pod="ai-tenant-e2e-shared-b-068f74/rate-test-model-068f74-kserve-d848dd5c9-sjv95"
Jul 07 12:08:44.608572 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:44.608393    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/222efa62-cd81-463f-b18d-474214fd2c71-model-cache\") pod \"rate-test-model-068f74-kserve-d848dd5c9-sjv95\" (UID: \"222efa62-cd81-463f-b18d-474214fd2c71\") " pod="ai-tenant-e2e-shared-b-068f74/rate-test-model-068f74-kserve-d848dd5c9-sjv95"
Jul 07 12:08:44.608572 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:44.608454    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/222efa62-cd81-463f-b18d-474214fd2c71-home\") pod \"rate-test-model-068f74-kserve-d848dd5c9-sjv95\" (UID: \"222efa62-cd81-463f-b18d-474214fd2c71\") " pod="ai-tenant-e2e-shared-b-068f74/rate-test-model-068f74-kserve-d848dd5c9-sjv95"
Jul 07 12:08:44.709772 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:44.709734    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/222efa62-cd81-463f-b18d-474214fd2c71-tls-certs\") pod \"rate-test-model-068f74-kserve-d848dd5c9-sjv95\" (UID: \"222efa62-cd81-463f-b18d-474214fd2c71\") " pod="ai-tenant-e2e-shared-b-068f74/rate-test-model-068f74-kserve-d848dd5c9-sjv95"
Jul 07 12:08:44.709772 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:44.709774    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/222efa62-cd81-463f-b18d-474214fd2c71-dshm\") pod \"rate-test-model-068f74-kserve-d848dd5c9-sjv95\" (UID: \"222efa62-cd81-463f-b18d-474214fd2c71\") " pod="ai-tenant-e2e-shared-b-068f74/rate-test-model-068f74-kserve-d848dd5c9-sjv95"
Jul 07 12:08:44.710048 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:44.709917    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/222efa62-cd81-463f-b18d-474214fd2c71-model-cache\") pod \"rate-test-model-068f74-kserve-d848dd5c9-sjv95\" (UID: \"222efa62-cd81-463f-b18d-474214fd2c71\") " pod="ai-tenant-e2e-shared-b-068f74/rate-test-model-068f74-kserve-d848dd5c9-sjv95"
Jul 07 12:08:44.710048 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:44.709992    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/222efa62-cd81-463f-b18d-474214fd2c71-home\") pod \"rate-test-model-068f74-kserve-d848dd5c9-sjv95\" (UID: \"222efa62-cd81-463f-b18d-474214fd2c71\") " pod="ai-tenant-e2e-shared-b-068f74/rate-test-model-068f74-kserve-d848dd5c9-sjv95"
Jul 07 12:08:44.710155 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:44.710071    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/222efa62-cd81-463f-b18d-474214fd2c71-tmp-dir\") pod \"rate-test-model-068f74-kserve-d848dd5c9-sjv95\" (UID: \"222efa62-cd81-463f-b18d-474214fd2c71\") " pod="ai-tenant-e2e-shared-b-068f74/rate-test-model-068f74-kserve-d848dd5c9-sjv95"
Jul 07 12:08:44.710155 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:44.710111    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g28wl\" (UniqueName: \"kubernetes.io/projected/222efa62-cd81-463f-b18d-474214fd2c71-kube-api-access-g28wl\") pod \"rate-test-model-068f74-kserve-d848dd5c9-sjv95\" (UID: \"222efa62-cd81-463f-b18d-474214fd2c71\") " pod="ai-tenant-e2e-shared-b-068f74/rate-test-model-068f74-kserve-d848dd5c9-sjv95"
Jul 07 12:08:44.710341 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:44.710316    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/222efa62-cd81-463f-b18d-474214fd2c71-model-cache\") pod \"rate-test-model-068f74-kserve-d848dd5c9-sjv95\" (UID: \"222efa62-cd81-463f-b18d-474214fd2c71\") " pod="ai-tenant-e2e-shared-b-068f74/rate-test-model-068f74-kserve-d848dd5c9-sjv95"
Jul 07 12:08:44.710401 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:44.710341    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/222efa62-cd81-463f-b18d-474214fd2c71-home\") pod \"rate-test-model-068f74-kserve-d848dd5c9-sjv95\" (UID: \"222efa62-cd81-463f-b18d-474214fd2c71\") " pod="ai-tenant-e2e-shared-b-068f74/rate-test-model-068f74-kserve-d848dd5c9-sjv95"
Jul 07 12:08:44.710453 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:44.710430    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/222efa62-cd81-463f-b18d-474214fd2c71-tmp-dir\") pod \"rate-test-model-068f74-kserve-d848dd5c9-sjv95\" (UID: \"222efa62-cd81-463f-b18d-474214fd2c71\") " pod="ai-tenant-e2e-shared-b-068f74/rate-test-model-068f74-kserve-d848dd5c9-sjv95"
Jul 07 12:08:44.712580 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:44.712560    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/222efa62-cd81-463f-b18d-474214fd2c71-dshm\") pod \"rate-test-model-068f74-kserve-d848dd5c9-sjv95\" (UID: \"222efa62-cd81-463f-b18d-474214fd2c71\") " pod="ai-tenant-e2e-shared-b-068f74/rate-test-model-068f74-kserve-d848dd5c9-sjv95"
Jul 07 12:08:44.712779 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:44.712762    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/222efa62-cd81-463f-b18d-474214fd2c71-tls-certs\") pod \"rate-test-model-068f74-kserve-d848dd5c9-sjv95\" (UID: \"222efa62-cd81-463f-b18d-474214fd2c71\") " pod="ai-tenant-e2e-shared-b-068f74/rate-test-model-068f74-kserve-d848dd5c9-sjv95"
Jul 07 12:08:44.718643 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:44.718617    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g28wl\" (UniqueName: \"kubernetes.io/projected/222efa62-cd81-463f-b18d-474214fd2c71-kube-api-access-g28wl\") pod \"rate-test-model-068f74-kserve-d848dd5c9-sjv95\" (UID: \"222efa62-cd81-463f-b18d-474214fd2c71\") " pod="ai-tenant-e2e-shared-b-068f74/rate-test-model-068f74-kserve-d848dd5c9-sjv95"
Jul 07 12:08:44.841286 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:44.841186    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-068f74/rate-test-model-068f74-kserve-d848dd5c9-sjv95"
Jul 07 12:08:44.979341 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:44.979309    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-068f74/rate-test-model-068f74-kserve-d848dd5c9-sjv95"]
Jul 07 12:08:44.981239 ip-10-0-135-78 kubenswrapper[2570]: W0707 12:08:44.981181    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod222efa62_cd81_463f_b18d_474214fd2c71.slice/crio-a96a71ef79cd2622d137b86b9851ca508fea51dd679b569a9dcac3ff18458ff9 WatchSource:0}: Error finding container a96a71ef79cd2622d137b86b9851ca508fea51dd679b569a9dcac3ff18458ff9: Status 404 returned error can't find the container with id a96a71ef79cd2622d137b86b9851ca508fea51dd679b569a9dcac3ff18458ff9
Jul 07 12:08:45.237455 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:45.237416    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-068f74/rate-test-model-068f74-kserve-d848dd5c9-sjv95" event={"ID":"222efa62-cd81-463f-b18d-474214fd2c71","Type":"ContainerStarted","Data":"e73604dd7a53c847d1132d84a02248d096452e901d0e8f178c8c39690c7b61d3"}
Jul 07 12:08:45.237455 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:45.237461    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-068f74/rate-test-model-068f74-kserve-d848dd5c9-sjv95" event={"ID":"222efa62-cd81-463f-b18d-474214fd2c71","Type":"ContainerStarted","Data":"a96a71ef79cd2622d137b86b9851ca508fea51dd679b569a9dcac3ff18458ff9"}
Jul 07 12:08:45.257966 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:45.257908    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-068f74/rate-test-model-068f74-kserve-d848dd5c9-sjv95" podStartSLOduration=1.257890632 podStartE2EDuration="1.257890632s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 12:08:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:08:45.255946378 +0000 UTC m=+2513.229586818" watchObservedRunningTime="2026-07-07 12:08:45.257890632 +0000 UTC m=+2513.231531042"
Jul 07 12:08:54.842248 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:54.842187    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-068f74/rate-test-model-068f74-kserve-d848dd5c9-sjv95"
Jul 07 12:08:54.842670 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:54.842264    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-068f74/rate-test-model-068f74-kserve-d848dd5c9-sjv95"
Jul 07 12:08:54.855007 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:54.854980    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-068f74/rate-test-model-068f74-kserve-d848dd5c9-sjv95"
Jul 07 12:08:55.302505 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:08:55.302472    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-068f74/rate-test-model-068f74-kserve-d848dd5c9-sjv95"
Jul 07 12:09:27.728847 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:27.728802    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-10f898/rate-test-model-10f898-kserve-648f468475-49ppf"]
Jul 07 12:09:28.734643 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:28.734600    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-068f74/rate-test-model-068f74-kserve-d848dd5c9-sjv95"]
Jul 07 12:09:42.748800 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:42.748746    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-10f898/rate-test-model-10f898-kserve-648f468475-49ppf" podUID="250f3d9c-4146-404e-80dc-caa57983e053" containerName="main" containerID="cri-o://aa08632ec69cac674821c058229779d28e05ba5d77cbd3a1645bd9941c21e97c" gracePeriod=45
Jul 07 12:09:42.997403 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:42.997377    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-10f898/rate-test-model-10f898-kserve-648f468475-49ppf"
Jul 07 12:09:43.083639 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:43.083545    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xgpn9\" (UniqueName: \"kubernetes.io/projected/250f3d9c-4146-404e-80dc-caa57983e053-kube-api-access-xgpn9\") pod \"250f3d9c-4146-404e-80dc-caa57983e053\" (UID: \"250f3d9c-4146-404e-80dc-caa57983e053\") "
Jul 07 12:09:43.083639 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:43.083636    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/250f3d9c-4146-404e-80dc-caa57983e053-dshm\") pod \"250f3d9c-4146-404e-80dc-caa57983e053\" (UID: \"250f3d9c-4146-404e-80dc-caa57983e053\") "
Jul 07 12:09:43.083892 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:43.083688    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/250f3d9c-4146-404e-80dc-caa57983e053-tls-certs\") pod \"250f3d9c-4146-404e-80dc-caa57983e053\" (UID: \"250f3d9c-4146-404e-80dc-caa57983e053\") "
Jul 07 12:09:43.083892 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:43.083708    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/250f3d9c-4146-404e-80dc-caa57983e053-model-cache\") pod \"250f3d9c-4146-404e-80dc-caa57983e053\" (UID: \"250f3d9c-4146-404e-80dc-caa57983e053\") "
Jul 07 12:09:43.083892 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:43.083742    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/250f3d9c-4146-404e-80dc-caa57983e053-home\") pod \"250f3d9c-4146-404e-80dc-caa57983e053\" (UID: \"250f3d9c-4146-404e-80dc-caa57983e053\") "
Jul 07 12:09:43.083892 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:43.083762    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/250f3d9c-4146-404e-80dc-caa57983e053-tmp-dir\") pod \"250f3d9c-4146-404e-80dc-caa57983e053\" (UID: \"250f3d9c-4146-404e-80dc-caa57983e053\") "
Jul 07 12:09:43.084123 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:43.084011    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/250f3d9c-4146-404e-80dc-caa57983e053-home" (OuterVolumeSpecName: "home") pod "250f3d9c-4146-404e-80dc-caa57983e053" (UID: "250f3d9c-4146-404e-80dc-caa57983e053"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:09:43.084123 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:43.084029    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/250f3d9c-4146-404e-80dc-caa57983e053-model-cache" (OuterVolumeSpecName: "model-cache") pod "250f3d9c-4146-404e-80dc-caa57983e053" (UID: "250f3d9c-4146-404e-80dc-caa57983e053"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:09:43.084123 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:43.084105    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/250f3d9c-4146-404e-80dc-caa57983e053-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "250f3d9c-4146-404e-80dc-caa57983e053" (UID: "250f3d9c-4146-404e-80dc-caa57983e053"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:09:43.085831 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:43.085800    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/250f3d9c-4146-404e-80dc-caa57983e053-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "250f3d9c-4146-404e-80dc-caa57983e053" (UID: "250f3d9c-4146-404e-80dc-caa57983e053"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 12:09:43.085949 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:43.085838    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/250f3d9c-4146-404e-80dc-caa57983e053-dshm" (OuterVolumeSpecName: "dshm") pod "250f3d9c-4146-404e-80dc-caa57983e053" (UID: "250f3d9c-4146-404e-80dc-caa57983e053"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:09:43.085949 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:43.085845    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/250f3d9c-4146-404e-80dc-caa57983e053-kube-api-access-xgpn9" (OuterVolumeSpecName: "kube-api-access-xgpn9") pod "250f3d9c-4146-404e-80dc-caa57983e053" (UID: "250f3d9c-4146-404e-80dc-caa57983e053"). InnerVolumeSpecName "kube-api-access-xgpn9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 12:09:43.185172 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:43.185137    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/250f3d9c-4146-404e-80dc-caa57983e053-tls-certs\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:09:43.185172 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:43.185164    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/250f3d9c-4146-404e-80dc-caa57983e053-model-cache\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:09:43.185172 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:43.185173    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/250f3d9c-4146-404e-80dc-caa57983e053-home\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:09:43.185419 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:43.185182    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/250f3d9c-4146-404e-80dc-caa57983e053-tmp-dir\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:09:43.185419 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:43.185190    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xgpn9\" (UniqueName: \"kubernetes.io/projected/250f3d9c-4146-404e-80dc-caa57983e053-kube-api-access-xgpn9\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:09:43.185419 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:43.185199    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/250f3d9c-4146-404e-80dc-caa57983e053-dshm\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:09:43.544848 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:43.544811    2570 generic.go:356] "Generic (PLEG): container finished" podID="250f3d9c-4146-404e-80dc-caa57983e053" containerID="aa08632ec69cac674821c058229779d28e05ba5d77cbd3a1645bd9941c21e97c" exitCode=0
Jul 07 12:09:43.544848 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:43.544849    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-10f898/rate-test-model-10f898-kserve-648f468475-49ppf"
Jul 07 12:09:43.545084 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:43.544891    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-10f898/rate-test-model-10f898-kserve-648f468475-49ppf" event={"ID":"250f3d9c-4146-404e-80dc-caa57983e053","Type":"ContainerDied","Data":"aa08632ec69cac674821c058229779d28e05ba5d77cbd3a1645bd9941c21e97c"}
Jul 07 12:09:43.545084 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:43.544935    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-10f898/rate-test-model-10f898-kserve-648f468475-49ppf" event={"ID":"250f3d9c-4146-404e-80dc-caa57983e053","Type":"ContainerDied","Data":"ef8e0ce0a7a32b3fcf1f1fb2e304b90c8ea43941cd8f26de1577fe2544616b44"}
Jul 07 12:09:43.545084 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:43.544953    2570 scope.go:117] "RemoveContainer" containerID="aa08632ec69cac674821c058229779d28e05ba5d77cbd3a1645bd9941c21e97c"
Jul 07 12:09:43.554411 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:43.554394    2570 scope.go:117] "RemoveContainer" containerID="aa08632ec69cac674821c058229779d28e05ba5d77cbd3a1645bd9941c21e97c"
Jul 07 12:09:43.554685 ip-10-0-135-78 kubenswrapper[2570]: E0707 12:09:43.554667    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa08632ec69cac674821c058229779d28e05ba5d77cbd3a1645bd9941c21e97c\": container with ID starting with aa08632ec69cac674821c058229779d28e05ba5d77cbd3a1645bd9941c21e97c not found: ID does not exist" containerID="aa08632ec69cac674821c058229779d28e05ba5d77cbd3a1645bd9941c21e97c"
Jul 07 12:09:43.554759 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:43.554693    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa08632ec69cac674821c058229779d28e05ba5d77cbd3a1645bd9941c21e97c"} err="failed to get container status \"aa08632ec69cac674821c058229779d28e05ba5d77cbd3a1645bd9941c21e97c\": rpc error: code = NotFound desc = could not find container \"aa08632ec69cac674821c058229779d28e05ba5d77cbd3a1645bd9941c21e97c\": container with ID starting with aa08632ec69cac674821c058229779d28e05ba5d77cbd3a1645bd9941c21e97c not found: ID does not exist"
Jul 07 12:09:43.569772 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:43.569751    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-10f898/rate-test-model-10f898-kserve-648f468475-49ppf"]
Jul 07 12:09:43.573125 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:43.573107    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-10f898/rate-test-model-10f898-kserve-648f468475-49ppf"]
Jul 07 12:09:43.754817 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:43.754779    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-068f74/rate-test-model-068f74-kserve-d848dd5c9-sjv95" podUID="222efa62-cd81-463f-b18d-474214fd2c71" containerName="main" containerID="cri-o://e73604dd7a53c847d1132d84a02248d096452e901d0e8f178c8c39690c7b61d3" gracePeriod=45
Jul 07 12:09:43.940267 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:43.940230    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="250f3d9c-4146-404e-80dc-caa57983e053" path="/var/lib/kubelet/pods/250f3d9c-4146-404e-80dc-caa57983e053/volumes"
Jul 07 12:09:44.009061 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:44.009033    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-068f74/rate-test-model-068f74-kserve-d848dd5c9-sjv95"
Jul 07 12:09:44.093086 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:44.093048    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/222efa62-cd81-463f-b18d-474214fd2c71-home\") pod \"222efa62-cd81-463f-b18d-474214fd2c71\" (UID: \"222efa62-cd81-463f-b18d-474214fd2c71\") "
Jul 07 12:09:44.093086 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:44.093085    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/222efa62-cd81-463f-b18d-474214fd2c71-dshm\") pod \"222efa62-cd81-463f-b18d-474214fd2c71\" (UID: \"222efa62-cd81-463f-b18d-474214fd2c71\") "
Jul 07 12:09:44.093353 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:44.093113    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/222efa62-cd81-463f-b18d-474214fd2c71-tls-certs\") pod \"222efa62-cd81-463f-b18d-474214fd2c71\" (UID: \"222efa62-cd81-463f-b18d-474214fd2c71\") "
Jul 07 12:09:44.093353 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:44.093180    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g28wl\" (UniqueName: \"kubernetes.io/projected/222efa62-cd81-463f-b18d-474214fd2c71-kube-api-access-g28wl\") pod \"222efa62-cd81-463f-b18d-474214fd2c71\" (UID: \"222efa62-cd81-463f-b18d-474214fd2c71\") "
Jul 07 12:09:44.093353 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:44.093246    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/222efa62-cd81-463f-b18d-474214fd2c71-model-cache\") pod \"222efa62-cd81-463f-b18d-474214fd2c71\" (UID: \"222efa62-cd81-463f-b18d-474214fd2c71\") "
Jul 07 12:09:44.093353 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:44.093311    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/222efa62-cd81-463f-b18d-474214fd2c71-tmp-dir\") pod \"222efa62-cd81-463f-b18d-474214fd2c71\" (UID: \"222efa62-cd81-463f-b18d-474214fd2c71\") "
Jul 07 12:09:44.093567 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:44.093346    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/222efa62-cd81-463f-b18d-474214fd2c71-home" (OuterVolumeSpecName: "home") pod "222efa62-cd81-463f-b18d-474214fd2c71" (UID: "222efa62-cd81-463f-b18d-474214fd2c71"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:09:44.093635 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:44.093561    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/222efa62-cd81-463f-b18d-474214fd2c71-model-cache" (OuterVolumeSpecName: "model-cache") pod "222efa62-cd81-463f-b18d-474214fd2c71" (UID: "222efa62-cd81-463f-b18d-474214fd2c71"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:09:44.093635 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:44.093586    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/222efa62-cd81-463f-b18d-474214fd2c71-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "222efa62-cd81-463f-b18d-474214fd2c71" (UID: "222efa62-cd81-463f-b18d-474214fd2c71"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:09:44.093751 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:44.093706    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/222efa62-cd81-463f-b18d-474214fd2c71-model-cache\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:09:44.093751 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:44.093727    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/222efa62-cd81-463f-b18d-474214fd2c71-tmp-dir\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:09:44.093751 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:44.093741    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/222efa62-cd81-463f-b18d-474214fd2c71-home\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:09:44.095326 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:44.095304    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/222efa62-cd81-463f-b18d-474214fd2c71-dshm" (OuterVolumeSpecName: "dshm") pod "222efa62-cd81-463f-b18d-474214fd2c71" (UID: "222efa62-cd81-463f-b18d-474214fd2c71"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:09:44.095441 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:44.095382    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/222efa62-cd81-463f-b18d-474214fd2c71-kube-api-access-g28wl" (OuterVolumeSpecName: "kube-api-access-g28wl") pod "222efa62-cd81-463f-b18d-474214fd2c71" (UID: "222efa62-cd81-463f-b18d-474214fd2c71"). InnerVolumeSpecName "kube-api-access-g28wl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 12:09:44.095441 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:44.095425    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/222efa62-cd81-463f-b18d-474214fd2c71-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "222efa62-cd81-463f-b18d-474214fd2c71" (UID: "222efa62-cd81-463f-b18d-474214fd2c71"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 12:09:44.194545 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:44.194507    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/222efa62-cd81-463f-b18d-474214fd2c71-dshm\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:09:44.194545 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:44.194540    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/222efa62-cd81-463f-b18d-474214fd2c71-tls-certs\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:09:44.194739 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:44.194555    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g28wl\" (UniqueName: \"kubernetes.io/projected/222efa62-cd81-463f-b18d-474214fd2c71-kube-api-access-g28wl\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:09:44.554433 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:44.554340    2570 generic.go:356] "Generic (PLEG): container finished" podID="222efa62-cd81-463f-b18d-474214fd2c71" containerID="e73604dd7a53c847d1132d84a02248d096452e901d0e8f178c8c39690c7b61d3" exitCode=0
Jul 07 12:09:44.554433 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:44.554402    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-068f74/rate-test-model-068f74-kserve-d848dd5c9-sjv95"
Jul 07 12:09:44.554433 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:44.554406    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-068f74/rate-test-model-068f74-kserve-d848dd5c9-sjv95" event={"ID":"222efa62-cd81-463f-b18d-474214fd2c71","Type":"ContainerDied","Data":"e73604dd7a53c847d1132d84a02248d096452e901d0e8f178c8c39690c7b61d3"}
Jul 07 12:09:44.554656 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:44.554444    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-068f74/rate-test-model-068f74-kserve-d848dd5c9-sjv95" event={"ID":"222efa62-cd81-463f-b18d-474214fd2c71","Type":"ContainerDied","Data":"a96a71ef79cd2622d137b86b9851ca508fea51dd679b569a9dcac3ff18458ff9"}
Jul 07 12:09:44.554656 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:44.554465    2570 scope.go:117] "RemoveContainer" containerID="e73604dd7a53c847d1132d84a02248d096452e901d0e8f178c8c39690c7b61d3"
Jul 07 12:09:44.570253 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:44.570228    2570 scope.go:117] "RemoveContainer" containerID="e73604dd7a53c847d1132d84a02248d096452e901d0e8f178c8c39690c7b61d3"
Jul 07 12:09:44.570510 ip-10-0-135-78 kubenswrapper[2570]: E0707 12:09:44.570490    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e73604dd7a53c847d1132d84a02248d096452e901d0e8f178c8c39690c7b61d3\": container with ID starting with e73604dd7a53c847d1132d84a02248d096452e901d0e8f178c8c39690c7b61d3 not found: ID does not exist" containerID="e73604dd7a53c847d1132d84a02248d096452e901d0e8f178c8c39690c7b61d3"
Jul 07 12:09:44.570564 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:44.570519    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e73604dd7a53c847d1132d84a02248d096452e901d0e8f178c8c39690c7b61d3"} err="failed to get container status \"e73604dd7a53c847d1132d84a02248d096452e901d0e8f178c8c39690c7b61d3\": rpc error: code = NotFound desc = could not find container \"e73604dd7a53c847d1132d84a02248d096452e901d0e8f178c8c39690c7b61d3\": container with ID starting with e73604dd7a53c847d1132d84a02248d096452e901d0e8f178c8c39690c7b61d3 not found: ID does not exist"
Jul 07 12:09:44.581887 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:44.581859    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-068f74/rate-test-model-068f74-kserve-d848dd5c9-sjv95"]
Jul 07 12:09:44.585618 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:44.585595    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-068f74/rate-test-model-068f74-kserve-d848dd5c9-sjv95"]
Jul 07 12:09:45.937750 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:09:45.937711    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="222efa62-cd81-463f-b18d-474214fd2c71" path="/var/lib/kubelet/pods/222efa62-cd81-463f-b18d-474214fd2c71/volumes"
Jul 07 12:10:19.647277 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:19.647174    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp"]
Jul 07 12:10:19.647877 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:19.647446    2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp" podUID="17652d42-c3a2-4a55-aa49-08067583e9a8" containerName="istio-proxy" containerID="cri-o://098f6a4313b21a43fa7527cfb3feff541e1c8722f3591d2903cd30ab3f06c61b" gracePeriod=30
Jul 07 12:10:24.449903 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:24.449867    2570 patch_prober.go:28] interesting pod/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 07 12:10:24.450343 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:24.449922    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp" podUID="17652d42-c3a2-4a55-aa49-08067583e9a8" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 12:10:24.772075 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:24.772039    2570 generic.go:356] "Generic (PLEG): container finished" podID="17652d42-c3a2-4a55-aa49-08067583e9a8" containerID="098f6a4313b21a43fa7527cfb3feff541e1c8722f3591d2903cd30ab3f06c61b" exitCode=0
Jul 07 12:10:24.772075 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:24.772067    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp" event={"ID":"17652d42-c3a2-4a55-aa49-08067583e9a8","Type":"ContainerDied","Data":"098f6a4313b21a43fa7527cfb3feff541e1c8722f3591d2903cd30ab3f06c61b"}
Jul 07 12:10:24.772267 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:24.772133    2570 scope.go:117] "RemoveContainer" containerID="f9b3e5829928345ddfac5463c2242b07c6d33ab8a83190045614c5428f44b447"
Jul 07 12:10:24.906577 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:24.906551    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp"
Jul 07 12:10:25.082546 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:25.082455    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17652d42-c3a2-4a55-aa49-08067583e9a8-workload-certs\") pod \"17652d42-c3a2-4a55-aa49-08067583e9a8\" (UID: \"17652d42-c3a2-4a55-aa49-08067583e9a8\") "
Jul 07 12:10:25.082546 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:25.082496    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8wjlz\" (UniqueName: \"kubernetes.io/projected/17652d42-c3a2-4a55-aa49-08067583e9a8-kube-api-access-8wjlz\") pod \"17652d42-c3a2-4a55-aa49-08067583e9a8\" (UID: \"17652d42-c3a2-4a55-aa49-08067583e9a8\") "
Jul 07 12:10:25.082546 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:25.082526    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17652d42-c3a2-4a55-aa49-08067583e9a8-istio-podinfo\") pod \"17652d42-c3a2-4a55-aa49-08067583e9a8\" (UID: \"17652d42-c3a2-4a55-aa49-08067583e9a8\") "
Jul 07 12:10:25.082856 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:25.082703    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17652d42-c3a2-4a55-aa49-08067583e9a8-istio-token\") pod \"17652d42-c3a2-4a55-aa49-08067583e9a8\" (UID: \"17652d42-c3a2-4a55-aa49-08067583e9a8\") "
Jul 07 12:10:25.082856 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:25.082762    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17652d42-c3a2-4a55-aa49-08067583e9a8-workload-socket\") pod \"17652d42-c3a2-4a55-aa49-08067583e9a8\" (UID: \"17652d42-c3a2-4a55-aa49-08067583e9a8\") "
Jul 07 12:10:25.082856 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:25.082834    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17652d42-c3a2-4a55-aa49-08067583e9a8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "17652d42-c3a2-4a55-aa49-08067583e9a8" (UID: "17652d42-c3a2-4a55-aa49-08067583e9a8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:10:25.083003 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:25.082882    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17652d42-c3a2-4a55-aa49-08067583e9a8-credential-socket\") pod \"17652d42-c3a2-4a55-aa49-08067583e9a8\" (UID: \"17652d42-c3a2-4a55-aa49-08067583e9a8\") "
Jul 07 12:10:25.083003 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:25.082928    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17652d42-c3a2-4a55-aa49-08067583e9a8-istiod-ca-cert\") pod \"17652d42-c3a2-4a55-aa49-08067583e9a8\" (UID: \"17652d42-c3a2-4a55-aa49-08067583e9a8\") "
Jul 07 12:10:25.083003 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:25.082951    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17652d42-c3a2-4a55-aa49-08067583e9a8-istio-envoy\") pod \"17652d42-c3a2-4a55-aa49-08067583e9a8\" (UID: \"17652d42-c3a2-4a55-aa49-08067583e9a8\") "
Jul 07 12:10:25.083003 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:25.082990    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17652d42-c3a2-4a55-aa49-08067583e9a8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "17652d42-c3a2-4a55-aa49-08067583e9a8" (UID: "17652d42-c3a2-4a55-aa49-08067583e9a8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:10:25.083242 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:25.083050    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17652d42-c3a2-4a55-aa49-08067583e9a8-istio-data\") pod \"17652d42-c3a2-4a55-aa49-08067583e9a8\" (UID: \"17652d42-c3a2-4a55-aa49-08067583e9a8\") "
Jul 07 12:10:25.083242 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:25.083158    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17652d42-c3a2-4a55-aa49-08067583e9a8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "17652d42-c3a2-4a55-aa49-08067583e9a8" (UID: "17652d42-c3a2-4a55-aa49-08067583e9a8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:10:25.083745 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:25.083313    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17652d42-c3a2-4a55-aa49-08067583e9a8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "17652d42-c3a2-4a55-aa49-08067583e9a8" (UID: "17652d42-c3a2-4a55-aa49-08067583e9a8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 12:10:25.083861 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:25.083482    2570 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17652d42-c3a2-4a55-aa49-08067583e9a8-workload-certs\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:10:25.083861 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:25.083775    2570 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17652d42-c3a2-4a55-aa49-08067583e9a8-workload-socket\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:10:25.083861 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:25.083791    2570 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17652d42-c3a2-4a55-aa49-08067583e9a8-credential-socket\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:10:25.084016 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:25.083961    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17652d42-c3a2-4a55-aa49-08067583e9a8-istio-data" (OuterVolumeSpecName: "istio-data") pod "17652d42-c3a2-4a55-aa49-08067583e9a8" (UID: "17652d42-c3a2-4a55-aa49-08067583e9a8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:10:25.084967 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:25.084943    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/17652d42-c3a2-4a55-aa49-08067583e9a8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "17652d42-c3a2-4a55-aa49-08067583e9a8" (UID: "17652d42-c3a2-4a55-aa49-08067583e9a8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 12:10:25.085179 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:25.085159    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17652d42-c3a2-4a55-aa49-08067583e9a8-kube-api-access-8wjlz" (OuterVolumeSpecName: "kube-api-access-8wjlz") pod "17652d42-c3a2-4a55-aa49-08067583e9a8" (UID: "17652d42-c3a2-4a55-aa49-08067583e9a8"). InnerVolumeSpecName "kube-api-access-8wjlz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 12:10:25.085264 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:25.085175    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17652d42-c3a2-4a55-aa49-08067583e9a8-istio-token" (OuterVolumeSpecName: "istio-token") pod "17652d42-c3a2-4a55-aa49-08067583e9a8" (UID: "17652d42-c3a2-4a55-aa49-08067583e9a8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 12:10:25.085387 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:25.085362    2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17652d42-c3a2-4a55-aa49-08067583e9a8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "17652d42-c3a2-4a55-aa49-08067583e9a8" (UID: "17652d42-c3a2-4a55-aa49-08067583e9a8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 12:10:25.185394 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:25.185351    2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17652d42-c3a2-4a55-aa49-08067583e9a8-istio-token\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:10:25.185394 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:25.185391    2570 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17652d42-c3a2-4a55-aa49-08067583e9a8-istiod-ca-cert\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:10:25.185394 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:25.185402    2570 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17652d42-c3a2-4a55-aa49-08067583e9a8-istio-envoy\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:10:25.185704 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:25.185413    2570 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17652d42-c3a2-4a55-aa49-08067583e9a8-istio-data\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:10:25.185704 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:25.185427    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8wjlz\" (UniqueName: \"kubernetes.io/projected/17652d42-c3a2-4a55-aa49-08067583e9a8-kube-api-access-8wjlz\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:10:25.185704 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:25.185442    2570 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17652d42-c3a2-4a55-aa49-08067583e9a8-istio-podinfo\") on node \"ip-10-0-135-78.ec2.internal\" DevicePath \"\""
Jul 07 12:10:25.779051 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:25.779022    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp"
Jul 07 12:10:25.779568 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:25.779064    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp" event={"ID":"17652d42-c3a2-4a55-aa49-08067583e9a8","Type":"ContainerDied","Data":"c5bd165c84617ed1a84ad59c2127921ff948db1f50ba1899b03bbce4915096c7"}
Jul 07 12:10:25.779568 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:25.779125    2570 scope.go:117] "RemoveContainer" containerID="098f6a4313b21a43fa7527cfb3feff541e1c8722f3591d2903cd30ab3f06c61b"
Jul 07 12:10:25.787595 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:25.787307    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-7dl7x_eb7560d9-3b8c-4d4b-99dd-190e3fbbd1b8/server/0.log"
Jul 07 12:10:25.801828 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:25.801805    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp"]
Jul 07 12:10:25.805602 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:25.805578    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-068f74-openshift-default-db45cf64c-9zcvp"]
Jul 07 12:10:25.937966 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:25.937933    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17652d42-c3a2-4a55-aa49-08067583e9a8" path="/var/lib/kubelet/pods/17652d42-c3a2-4a55-aa49-08067583e9a8/volumes"
Jul 07 12:10:26.301104 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:26.301076    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f68d9489b-zlljb_c245d200-66ef-4446-809a-db8b19cb4b77/copy-manifests/0.log"
Jul 07 12:10:26.308306 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:26.308278    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f68d9489b-zlljb_c245d200-66ef-4446-809a-db8b19cb4b77/manager/0.log"
Jul 07 12:10:27.227660 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:27.227630    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h75kt_536b501c-5d9f-4b9b-9bcd-296787d774c7/util/0.log"
Jul 07 12:10:27.234399 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:27.234380    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h75kt_536b501c-5d9f-4b9b-9bcd-296787d774c7/pull/0.log"
Jul 07 12:10:27.240539 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:27.240516    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h75kt_536b501c-5d9f-4b9b-9bcd-296787d774c7/extract/0.log"
Jul 07 12:10:27.354766 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:27.354738    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cpcql_fce422a1-973a-4cd1-a894-a0f2e7706f8f/util/0.log"
Jul 07 12:10:27.361769 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:27.361743    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cpcql_fce422a1-973a-4cd1-a894-a0f2e7706f8f/pull/0.log"
Jul 07 12:10:27.367684 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:27.367661    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cpcql_fce422a1-973a-4cd1-a894-a0f2e7706f8f/extract/0.log"
Jul 07 12:10:27.488613 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:27.488522    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nd2r4_4d631fc5-7f71-4c37-bcd3-59a88dde0305/util/0.log"
Jul 07 12:10:27.495101 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:27.495080    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nd2r4_4d631fc5-7f71-4c37-bcd3-59a88dde0305/pull/0.log"
Jul 07 12:10:27.501279 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:27.501260    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nd2r4_4d631fc5-7f71-4c37-bcd3-59a88dde0305/extract/0.log"
Jul 07 12:10:27.621179 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:27.621153    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lns8w_4f247630-5817-4f35-8527-97024949c300/util/0.log"
Jul 07 12:10:27.627576 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:27.627529    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lns8w_4f247630-5817-4f35-8527-97024949c300/pull/0.log"
Jul 07 12:10:27.634163 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:27.634140    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lns8w_4f247630-5817-4f35-8527-97024949c300/extract/0.log"
Jul 07 12:10:28.913951 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:28.913915    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf48fwzr_8abca133-c3a3-47eb-98d1-acaa6743e025/istio-proxy/0.log"
Jul 07 12:10:29.141146 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:29.141121    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5474cb5f94-92m5v_9cf22451-0c1c-4c2b-b2f4-c9dc6ff88f06/kube-auth-proxy/0.log"
Jul 07 12:10:29.372596 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:29.372560    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vjhwc_5b0afd6c-1e2a-4dee-b8fd-cde077988d1e/istio-proxy/1.log"
Jul 07 12:10:29.719893 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:29.719860    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5647bb9f4c-npcb5_331a680b-b6dd-46fd-b0da-cccfefd15bd3/router/0.log"
Jul 07 12:10:30.296420 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:30.296382    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-5kn7q_3838b1cc-3390-449b-b705-97fbb6c5caf2/main/0.log"
Jul 07 12:10:30.414475 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:30.414446    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrbbwp_14348839-2afe-4987-84c9-46ed1f0bb1a7/main/0.log"
Jul 07 12:10:30.667467 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:30.667381    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-45cgf_a19573b6-1ff3-4b0a-a47f-915c7aa2b490/main/0.log"
Jul 07 12:10:37.803790 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:37.803761    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-8f5jf_afdb1ba0-560a-4b0f-883b-aa960c6dd304/global-pull-secret-syncer/0.log"
Jul 07 12:10:37.928614 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:37.928582    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-7tk6n_af613b4a-9c2e-4b9b-8a10-0ca67f1f9159/konnectivity-agent/0.log"
Jul 07 12:10:38.002341 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:38.002313    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-78.ec2.internal_4f2009fc30615ef1da5c98996e493d3f/haproxy/0.log"
Jul 07 12:10:42.137550 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:42.137519    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h75kt_536b501c-5d9f-4b9b-9bcd-296787d774c7/extract/0.log"
Jul 07 12:10:42.164700 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:42.164658    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h75kt_536b501c-5d9f-4b9b-9bcd-296787d774c7/util/0.log"
Jul 07 12:10:42.186360 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:42.186329    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759h75kt_536b501c-5d9f-4b9b-9bcd-296787d774c7/pull/0.log"
Jul 07 12:10:42.220571 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:42.220533    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cpcql_fce422a1-973a-4cd1-a894-a0f2e7706f8f/extract/0.log"
Jul 07 12:10:42.247630 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:42.247603    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cpcql_fce422a1-973a-4cd1-a894-a0f2e7706f8f/util/0.log"
Jul 07 12:10:42.278852 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:42.278823    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cpcql_fce422a1-973a-4cd1-a894-a0f2e7706f8f/pull/0.log"
Jul 07 12:10:42.308259 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:42.308205    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nd2r4_4d631fc5-7f71-4c37-bcd3-59a88dde0305/extract/0.log"
Jul 07 12:10:42.337044 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:42.337015    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nd2r4_4d631fc5-7f71-4c37-bcd3-59a88dde0305/util/0.log"
Jul 07 12:10:42.361503 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:42.361425    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73nd2r4_4d631fc5-7f71-4c37-bcd3-59a88dde0305/pull/0.log"
Jul 07 12:10:42.390033 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:42.390000    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lns8w_4f247630-5817-4f35-8527-97024949c300/extract/0.log"
Jul 07 12:10:42.412364 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:42.412339    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lns8w_4f247630-5817-4f35-8527-97024949c300/util/0.log"
Jul 07 12:10:42.442409 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:42.442376    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1lns8w_4f247630-5817-4f35-8527-97024949c300/pull/0.log"
Jul 07 12:10:45.125374 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:45.125269    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-ncgwv_561f4302-647f-4dd2-bbcb-927a28c17df7/cluster-monitoring-operator/0.log"
Jul 07 12:10:45.154151 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:45.154116    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-j4488_f8e91d6e-7866-4576-bae6-a351d600d703/kube-state-metrics/0.log"
Jul 07 12:10:45.172137 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:45.172109    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-j4488_f8e91d6e-7866-4576-bae6-a351d600d703/kube-rbac-proxy-main/0.log"
Jul 07 12:10:45.192018 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:45.191977    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-j4488_f8e91d6e-7866-4576-bae6-a351d600d703/kube-rbac-proxy-self/0.log"
Jul 07 12:10:45.303637 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:45.303603    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b9rvb_54a5035f-302e-4f61-a39a-cf4eebf2f669/node-exporter/0.log"
Jul 07 12:10:45.324877 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:45.324848    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b9rvb_54a5035f-302e-4f61-a39a-cf4eebf2f669/kube-rbac-proxy/0.log"
Jul 07 12:10:45.363871 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:45.363846    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b9rvb_54a5035f-302e-4f61-a39a-cf4eebf2f669/init-textfile/0.log"
Jul 07 12:10:45.539155 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:45.539118    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-gwvbz_3e9e44ad-ca0e-4ae8-af94-0d51e0259ce2/kube-rbac-proxy-main/0.log"
Jul 07 12:10:45.560392 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:45.560361    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-gwvbz_3e9e44ad-ca0e-4ae8-af94-0d51e0259ce2/kube-rbac-proxy-self/0.log"
Jul 07 12:10:45.579776 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:45.579744    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-gwvbz_3e9e44ad-ca0e-4ae8-af94-0d51e0259ce2/openshift-state-metrics/0.log"
Jul 07 12:10:45.786168 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:45.786141    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-kf2j7_1fcc05da-8891-4f1f-bbb7-4e886316c910/prometheus-operator/0.log"
Jul 07 12:10:45.804725 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:45.804660    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-kf2j7_1fcc05da-8891-4f1f-bbb7-4e886316c910/kube-rbac-proxy/0.log"
Jul 07 12:10:45.831890 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:45.831864    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-c6mxv_ce03fa2b-19e6-407c-ad1b-474032191b71/prometheus-operator-admission-webhook/0.log"
Jul 07 12:10:47.388867 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:47.388837    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-wlk99_88a181af-4d7c-4398-a53b-69ab8f2e4035/networking-console-plugin/0.log"
Jul 07 12:10:48.006043 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:48.006015    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ktbpw_faf6f343-6515-4b2c-a34e-286cd2d9ec7a/console-operator/1.log"
Jul 07 12:10:48.014391 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:48.014365    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-ktbpw_faf6f343-6515-4b2c-a34e-286cd2d9ec7a/console-operator/2.log"
Jul 07 12:10:48.434034 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:48.434000    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-vbjbt/perf-node-gather-daemonset-2hdn8"]
Jul 07 12:10:48.434457 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:48.434355    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="222efa62-cd81-463f-b18d-474214fd2c71" containerName="main"
Jul 07 12:10:48.434457 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:48.434366    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="222efa62-cd81-463f-b18d-474214fd2c71" containerName="main"
Jul 07 12:10:48.434457 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:48.434376    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17652d42-c3a2-4a55-aa49-08067583e9a8" containerName="istio-proxy"
Jul 07 12:10:48.434457 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:48.434382    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="17652d42-c3a2-4a55-aa49-08067583e9a8" containerName="istio-proxy"
Jul 07 12:10:48.434457 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:48.434396    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17652d42-c3a2-4a55-aa49-08067583e9a8" containerName="istio-proxy"
Jul 07 12:10:48.434457 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:48.434402    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="17652d42-c3a2-4a55-aa49-08067583e9a8" containerName="istio-proxy"
Jul 07 12:10:48.434457 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:48.434409    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="250f3d9c-4146-404e-80dc-caa57983e053" containerName="main"
Jul 07 12:10:48.434457 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:48.434415    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="250f3d9c-4146-404e-80dc-caa57983e053" containerName="main"
Jul 07 12:10:48.434786 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:48.434471    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="250f3d9c-4146-404e-80dc-caa57983e053" containerName="main"
Jul 07 12:10:48.434786 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:48.434481    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="222efa62-cd81-463f-b18d-474214fd2c71" containerName="main"
Jul 07 12:10:48.434786 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:48.434487    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="17652d42-c3a2-4a55-aa49-08067583e9a8" containerName="istio-proxy"
Jul 07 12:10:48.434786 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:48.434493    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="17652d42-c3a2-4a55-aa49-08067583e9a8" containerName="istio-proxy"
Jul 07 12:10:48.437798 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:48.437777    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vbjbt/perf-node-gather-daemonset-2hdn8"
Jul 07 12:10:48.440752 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:48.440733    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vbjbt\"/\"kube-root-ca.crt\""
Jul 07 12:10:48.441844 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:48.441812    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-vbjbt\"/\"default-dockercfg-x8ms6\""
Jul 07 12:10:48.441844 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:48.441835    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vbjbt\"/\"openshift-service-ca.crt\""
Jul 07 12:10:48.447822 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:48.447797    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vbjbt/perf-node-gather-daemonset-2hdn8"]
Jul 07 12:10:48.484110 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:48.484080    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a1c56849-5374-4e77-aec5-7276253e3ebc-podres\") pod \"perf-node-gather-daemonset-2hdn8\" (UID: \"a1c56849-5374-4e77-aec5-7276253e3ebc\") " pod="openshift-must-gather-vbjbt/perf-node-gather-daemonset-2hdn8"
Jul 07 12:10:48.484306 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:48.484142    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a1c56849-5374-4e77-aec5-7276253e3ebc-lib-modules\") pod \"perf-node-gather-daemonset-2hdn8\" (UID: \"a1c56849-5374-4e77-aec5-7276253e3ebc\") " pod="openshift-must-gather-vbjbt/perf-node-gather-daemonset-2hdn8"
Jul 07 12:10:48.484306 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:48.484164    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lp2w7\" (UniqueName: \"kubernetes.io/projected/a1c56849-5374-4e77-aec5-7276253e3ebc-kube-api-access-lp2w7\") pod \"perf-node-gather-daemonset-2hdn8\" (UID: \"a1c56849-5374-4e77-aec5-7276253e3ebc\") " pod="openshift-must-gather-vbjbt/perf-node-gather-daemonset-2hdn8"
Jul 07 12:10:48.484306 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:48.484186    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a1c56849-5374-4e77-aec5-7276253e3ebc-proc\") pod \"perf-node-gather-daemonset-2hdn8\" (UID: \"a1c56849-5374-4e77-aec5-7276253e3ebc\") " pod="openshift-must-gather-vbjbt/perf-node-gather-daemonset-2hdn8"
Jul 07 12:10:48.484306 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:48.484229    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a1c56849-5374-4e77-aec5-7276253e3ebc-sys\") pod \"perf-node-gather-daemonset-2hdn8\" (UID: \"a1c56849-5374-4e77-aec5-7276253e3ebc\") " pod="openshift-must-gather-vbjbt/perf-node-gather-daemonset-2hdn8"
Jul 07 12:10:48.585301 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:48.585269    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a1c56849-5374-4e77-aec5-7276253e3ebc-lib-modules\") pod \"perf-node-gather-daemonset-2hdn8\" (UID: \"a1c56849-5374-4e77-aec5-7276253e3ebc\") " pod="openshift-must-gather-vbjbt/perf-node-gather-daemonset-2hdn8"
Jul 07 12:10:48.585301 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:48.585303    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lp2w7\" (UniqueName: \"kubernetes.io/projected/a1c56849-5374-4e77-aec5-7276253e3ebc-kube-api-access-lp2w7\") pod \"perf-node-gather-daemonset-2hdn8\" (UID: \"a1c56849-5374-4e77-aec5-7276253e3ebc\") " pod="openshift-must-gather-vbjbt/perf-node-gather-daemonset-2hdn8"
Jul 07 12:10:48.585525 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:48.585330    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a1c56849-5374-4e77-aec5-7276253e3ebc-proc\") pod \"perf-node-gather-daemonset-2hdn8\" (UID: \"a1c56849-5374-4e77-aec5-7276253e3ebc\") " pod="openshift-must-gather-vbjbt/perf-node-gather-daemonset-2hdn8"
Jul 07 12:10:48.585525 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:48.585351    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a1c56849-5374-4e77-aec5-7276253e3ebc-sys\") pod \"perf-node-gather-daemonset-2hdn8\" (UID: \"a1c56849-5374-4e77-aec5-7276253e3ebc\") " pod="openshift-must-gather-vbjbt/perf-node-gather-daemonset-2hdn8"
Jul 07 12:10:48.585525 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:48.585390    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a1c56849-5374-4e77-aec5-7276253e3ebc-podres\") pod \"perf-node-gather-daemonset-2hdn8\" (UID: \"a1c56849-5374-4e77-aec5-7276253e3ebc\") " pod="openshift-must-gather-vbjbt/perf-node-gather-daemonset-2hdn8"
Jul 07 12:10:48.585525 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:48.585441    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a1c56849-5374-4e77-aec5-7276253e3ebc-lib-modules\") pod \"perf-node-gather-daemonset-2hdn8\" (UID: \"a1c56849-5374-4e77-aec5-7276253e3ebc\") " pod="openshift-must-gather-vbjbt/perf-node-gather-daemonset-2hdn8"
Jul 07 12:10:48.585525 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:48.585480    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a1c56849-5374-4e77-aec5-7276253e3ebc-proc\") pod \"perf-node-gather-daemonset-2hdn8\" (UID: \"a1c56849-5374-4e77-aec5-7276253e3ebc\") " pod="openshift-must-gather-vbjbt/perf-node-gather-daemonset-2hdn8"
Jul 07 12:10:48.585525 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:48.585489    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a1c56849-5374-4e77-aec5-7276253e3ebc-sys\") pod \"perf-node-gather-daemonset-2hdn8\" (UID: \"a1c56849-5374-4e77-aec5-7276253e3ebc\") " pod="openshift-must-gather-vbjbt/perf-node-gather-daemonset-2hdn8"
Jul 07 12:10:48.585525 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:48.585497    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a1c56849-5374-4e77-aec5-7276253e3ebc-podres\") pod \"perf-node-gather-daemonset-2hdn8\" (UID: \"a1c56849-5374-4e77-aec5-7276253e3ebc\") " pod="openshift-must-gather-vbjbt/perf-node-gather-daemonset-2hdn8"
Jul 07 12:10:48.593657 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:48.593634    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lp2w7\" (UniqueName: \"kubernetes.io/projected/a1c56849-5374-4e77-aec5-7276253e3ebc-kube-api-access-lp2w7\") pod \"perf-node-gather-daemonset-2hdn8\" (UID: \"a1c56849-5374-4e77-aec5-7276253e3ebc\") " pod="openshift-must-gather-vbjbt/perf-node-gather-daemonset-2hdn8"
Jul 07 12:10:48.748914 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:48.748831    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vbjbt/perf-node-gather-daemonset-2hdn8"
Jul 07 12:10:48.879259 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:48.879223    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vbjbt/perf-node-gather-daemonset-2hdn8"]
Jul 07 12:10:48.880078 ip-10-0-135-78 kubenswrapper[2570]: W0707 12:10:48.880047    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda1c56849_5374_4e77_aec5_7276253e3ebc.slice/crio-8b3049f4579aa0f58bf548e179daeb938731656b5def919999a1e4830536ea0a WatchSource:0}: Error finding container 8b3049f4579aa0f58bf548e179daeb938731656b5def919999a1e4830536ea0a: Status 404 returned error can't find the container with id 8b3049f4579aa0f58bf548e179daeb938731656b5def919999a1e4830536ea0a
Jul 07 12:10:48.900139 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:48.900115    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vbjbt/perf-node-gather-daemonset-2hdn8" event={"ID":"a1c56849-5374-4e77-aec5-7276253e3ebc","Type":"ContainerStarted","Data":"8b3049f4579aa0f58bf548e179daeb938731656b5def919999a1e4830536ea0a"}
Jul 07 12:10:49.906748 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:49.906707    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vbjbt/perf-node-gather-daemonset-2hdn8" event={"ID":"a1c56849-5374-4e77-aec5-7276253e3ebc","Type":"ContainerStarted","Data":"6eae31abcd75c03d5b2a9887f2e743c728c1ca678909c7895b687a7f3f0b6cd6"}
Jul 07 12:10:49.907188 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:49.906856    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-vbjbt/perf-node-gather-daemonset-2hdn8"
Jul 07 12:10:49.923617 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:49.923570    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-vbjbt/perf-node-gather-daemonset-2hdn8" podStartSLOduration=1.923555871 podStartE2EDuration="1.923555871s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 12:10:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 12:10:49.921836061 +0000 UTC m=+2637.895476495" watchObservedRunningTime="2026-07-07 12:10:49.923555871 +0000 UTC m=+2637.897196284"
Jul 07 12:10:50.461023 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:50.460993    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9gqzt_bfcc5fcf-866f-405b-ae62-dcf401627bc7/dns/0.log"
Jul 07 12:10:50.482167 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:50.482138    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9gqzt_bfcc5fcf-866f-405b-ae62-dcf401627bc7/kube-rbac-proxy/0.log"
Jul 07 12:10:50.608333 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:50.608303    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rkn2t_650c2a15-194e-47f4-99c8-9f94802d1e9b/dns-node-resolver/0.log"
Jul 07 12:10:51.206355 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:51.206321    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qh2dm_e54ef86d-1806-4e84-a235-aba912f88f58/node-ca/0.log"
Jul 07 12:10:52.217981 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:52.217951    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf48fwzr_8abca133-c3a3-47eb-98d1-acaa6743e025/istio-proxy/0.log"
Jul 07 12:10:52.340941 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:52.340910    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5474cb5f94-92m5v_9cf22451-0c1c-4c2b-b2f4-c9dc6ff88f06/kube-auth-proxy/0.log"
Jul 07 12:10:52.395377 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:52.395326    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vjhwc_5b0afd6c-1e2a-4dee-b8fd-cde077988d1e/istio-proxy/0.log"
Jul 07 12:10:52.451003 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:52.450967    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vjhwc_5b0afd6c-1e2a-4dee-b8fd-cde077988d1e/istio-proxy/1.log"
Jul 07 12:10:52.565015 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:52.564930    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5647bb9f4c-npcb5_331a680b-b6dd-46fd-b0da-cccfefd15bd3/router/0.log"
Jul 07 12:10:53.307160 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:53.307131    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zd288_24d1f3b4-0970-41a7-95e6-6ab8e00f5b47/serve-healthcheck-canary/0.log"
Jul 07 12:10:53.766447 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:53.766419    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-ngjjc_49b59d5c-4db1-4013-8d2e-ac8690c4723c/insights-operator/0.log"
Jul 07 12:10:53.766653 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:53.766631    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-ngjjc_49b59d5c-4db1-4013-8d2e-ac8690c4723c/insights-operator/1.log"
Jul 07 12:10:53.925470 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:53.925431    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-r4q9n_1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32/kube-rbac-proxy/0.log"
Jul 07 12:10:53.951498 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:53.951469    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-r4q9n_1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32/exporter/0.log"
Jul 07 12:10:53.973990 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:53.973965    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-r4q9n_1b5d28c2-dcc8-4ad9-a0f4-5f36a5d67e32/extractor/0.log"
Jul 07 12:10:55.921225 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:55.921182    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-vbjbt/perf-node-gather-daemonset-2hdn8"
Jul 07 12:10:56.551450 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:56.551420    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-7dl7x_eb7560d9-3b8c-4d4b-99dd-190e3fbbd1b8/server/0.log"
Jul 07 12:10:56.803936 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:56.803852    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f68d9489b-zlljb_c245d200-66ef-4446-809a-db8b19cb4b77/manager/0.log"
Jul 07 12:10:56.826229 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:56.826181    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f68d9489b-zlljb_c245d200-66ef-4446-809a-db8b19cb4b77/copy-manifests/0.log"
Jul 07 12:10:58.357962 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:58.357923    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-69986dd77b-dqqh9_44444391-e155-47ac-b321-d421f0c55a52/manager/0.log"
Jul 07 12:10:58.972805 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:58.972773    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf48fwzr_8abca133-c3a3-47eb-98d1-acaa6743e025/istio-proxy/0.log"
Jul 07 12:10:59.084610 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:59.084561    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5474cb5f94-92m5v_9cf22451-0c1c-4c2b-b2f4-c9dc6ff88f06/kube-auth-proxy/0.log"
Jul 07 12:10:59.130808 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:59.130781    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vjhwc_5b0afd6c-1e2a-4dee-b8fd-cde077988d1e/istio-proxy/0.log"
Jul 07 12:10:59.182219 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:59.182189    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vjhwc_5b0afd6c-1e2a-4dee-b8fd-cde077988d1e/istio-proxy/1.log"
Jul 07 12:10:59.270986 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:10:59.270901    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5647bb9f4c-npcb5_331a680b-b6dd-46fd-b0da-cccfefd15bd3/router/0.log"
Jul 07 12:11:00.463105 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:11:00.463074    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf48fwzr_8abca133-c3a3-47eb-98d1-acaa6743e025/istio-proxy/0.log"
Jul 07 12:11:00.578380 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:11:00.578354    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5474cb5f94-92m5v_9cf22451-0c1c-4c2b-b2f4-c9dc6ff88f06/kube-auth-proxy/0.log"
Jul 07 12:11:00.619667 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:11:00.619631    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vjhwc_5b0afd6c-1e2a-4dee-b8fd-cde077988d1e/istio-proxy/0.log"
Jul 07 12:11:00.677570 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:11:00.677535    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vjhwc_5b0afd6c-1e2a-4dee-b8fd-cde077988d1e/istio-proxy/1.log"
Jul 07 12:11:00.781192 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:11:00.781104    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5647bb9f4c-npcb5_331a680b-b6dd-46fd-b0da-cccfefd15bd3/router/0.log"
Jul 07 12:11:05.069859 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:11:05.069775    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8f25d_20c2dc90-b7d3-4add-af1f-31f1ba991c54/kube-multus-additional-cni-plugins/0.log"
Jul 07 12:11:05.090830 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:11:05.090785    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8f25d_20c2dc90-b7d3-4add-af1f-31f1ba991c54/egress-router-binary-copy/0.log"
Jul 07 12:11:05.111108 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:11:05.111082    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8f25d_20c2dc90-b7d3-4add-af1f-31f1ba991c54/cni-plugins/0.log"
Jul 07 12:11:05.131525 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:11:05.131499    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8f25d_20c2dc90-b7d3-4add-af1f-31f1ba991c54/bond-cni-plugin/0.log"
Jul 07 12:11:05.150976 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:11:05.150947    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8f25d_20c2dc90-b7d3-4add-af1f-31f1ba991c54/routeoverride-cni/0.log"
Jul 07 12:11:05.171395 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:11:05.171365    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8f25d_20c2dc90-b7d3-4add-af1f-31f1ba991c54/whereabouts-cni-bincopy/0.log"
Jul 07 12:11:05.194432 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:11:05.194404    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8f25d_20c2dc90-b7d3-4add-af1f-31f1ba991c54/whereabouts-cni/0.log"
Jul 07 12:11:05.393112 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:11:05.393027    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-lj4rs_4e27418c-5ace-4a75-a4b9-931fffa64650/kube-multus/0.log"
Jul 07 12:11:05.522100 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:11:05.522067    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-ppmbz_f75e11d0-c0de-4a8d-849d-807a87aa1497/network-metrics-daemon/0.log"
Jul 07 12:11:05.539745 ip-10-0-135-78 kubenswrapper[2570]: I0707 12:11:05.539717    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-ppmbz_f75e11d0-c0de-4a8d-849d-807a87aa1497/kube-rbac-proxy/0.log"